What the Heck Does Scale Mean Anyway?

One of the core values of Xively has always been our ability to scale.  We talk about scalability as one of our key differentiators on why companies should look to us.  In fact, we recently did a benchmark test with Enterprise Strategy Group to validate the scale of the Xively platform and we are incredibly proud of the results.  ESG’s tests showed that we can support over 86 Billion messages a day on one Xively cluster, which is a huge number.  For comparison purposes – there are about 80 billion messages sent worldwide across Facebook Messenger, Whats App and SMS text each day.

But it occurred to me that scale means a lot of different things to different people, so I thought we should explore what scalability means in the IoT – why is it important and why it can be so hard to achieve.

What Does Scale Really Mean?

By definition scale refers to “the capability of a system, network, or process to handle a growing amount of work, or its potential to be enlarged in order to accommodate that growth”.  When you hear that definition you might translate it as “My system needs to be able to handle the data created by my current and future expected customer base”.  While that is certainly part of it, it’s just the tip of the iceberg.  It is true that when running a connected business (or really any business for that matter), you need to be able to service your millionth customer just as good as you did your first – latency threshold, connection time, linearly scaling infrastructure, meeting your SLAs, etc.  And moving the data associated with product usage is certainly one key factor, but there are others at play as well.   Areas like firmware updates, uptime, data management, identity and access management, and making the data actionable all play into scale.

And if you’re thinking a million customers is too large to fathom at this stage in the game – remember this, that’s less than 1% of American Households.

Challenges of Scale

If achieving scale was easy, we probably wouldn’t be having this conversation right now.  First of all, it’s not a one size fits all thing.  Some products need capacity for more messages than others and it can be hard to plan ahead for what you may need 2, 3, or even 5 years down the road.  Because of that, many connected product companies don’t think much further than their first POC and that can cause complicated issues later on.  Re-platforming is the equivalent of buying a larger house to fit a growing family.  It can be done, but it takes a lot of coordination and work to make the transition smooth.  And when you have a family of a million customers banking on you – it can also be really risky.  Which brings me to my next point…

Scalability helps reduce risk.  Plain and simple. There are so many things in the world of the IoT that product manufactures can’t control – user network, service provider latency, power or service outages.  Scalability is actually something we can control.  Planning ahead for the best case scenario of what you will need, the data models you may want, or the identity and access controls you will employ helps reduce the risk of not being able to meet customer expectations as your connected business grows.

I know this probably sounds like a pitch for over buying right?  That’s really not it at all.  It’s more about ensuring that your platform – whether you go to a vendor or build your own – is flexible enough for your future needs.  For example, let’s say you add a great new feature to your IoT product – one that your customers have been asking for and can’t wait to use.  That’s great, but now say that feature requires a significant increase in messages to work properly.  Knowing the infrastructure can scale to handle it is invaluable.   Or let’s say in the beginning you decide that you only want to collect messages that are going to let your support staff know about uptime of customers, but later your product team realizes that if they had information on what features were used most by customers it would better help inform the roadmap making the product more appealing.  That’s another group of messages the infrastructure not only needs to be able to collect, but manage but also potentially feed into business systems to make actionable.  Rolling the dice on whether it can handle this new request shouldn’t be an option.

Scalability is an absolute necessity for the success of the IoT.  Analyst firms and others are predicting that there will be billions of active connected products by 2020.  Whether accurate or not, we must prepare for it or risk becoming the victims of our own success.

Check out the ESG report to learn how Xively has cracked the scalability code.

Leave a comment

Leave a Reply

Explore our other Xively News or Recent posts.