Let’s start with the statement I made in the last posting:

Executing cpu instructions is what a data center does.

This is the point of the whole series of articles I’ve produced here. If you build a car factory it’s in order to produce cars (of a given quality) as economically as possible. If you build a power plant it’s in order to generate power as economically as possible. If you build a scaled out data center, it is in order to execute cpu instructions as economically as possible.

Building such assets is a risky business. If you’re in a competitive market of any kind you can only control the price to a limited degree.  After that, profitability depends on sales success and cost control.

So a company will build a massively scaled data center (or several) with the specific goal of keeping throughput costs as low as possible. It doesn’t matter whether it’s Infrastructure as a Service (IaaS), Platform as a Service (PaaS) or Software as a Service (SaaS), the metric that is going to matter is the cost of executing each instruction. Actually there are other metrics that matter, the cost of managing each byte of data stored and the cost of each byte of information transmitted or received by the data center, but they are the same kind of metric.

I’m deliberately framing this in technical terms, because if you are going to optimize the throughput of a data center, you need to look at every possible strategy and it starts with the nuts and bolts. In fact, it’s mostly about nuts and bolts. So I’ve created a nuts and bolts list of ten possible areas where you might have a technical axe to grind:

1. The cpu chips: Why not have your own chips designed? The division between machine instructions on the chip and software (compiled instructions sent to the chip) evolved from the need for flexibility, but in the scaled up data center you may not care much for flexibility at all. If it’s a PaaS data center or an SaaS data center, why not burn the application (or some of it) onto the chip. It will go a lot faster if you do.
2. The OS: Why not pay a team of software engineers to optimize the OS you’re using for your area of application. This is particularly easy if you use Linux of FreeBSD, because there are lots of software engineers around who know those environments. I discussed this a little in the last posting in respect of removing unnecessary bits. However you may wish to go a little further than that in terms of having an OS that is communicating with other OSes across a vast network of OSes. You might like to include specific scale out capabilities.
3. The App: Again for PaaS and SaaS, why not optimize the apps too. They might be written in an inefficient programming language; well rewrite in an efficient one. It might even be possible to tinker with the compiler or isolate routines that would be done better in Assembler and have someone translate them to a lower level. Remember that you’re building for a specific target environment.
4. Real 64 bit architecture. A 64 bit address space is very big (in the exabyte area.) It can certainly directly address all the memory on 50,000 or 100,000 servers, plus a vast array of solid state memory too. We don’t have OSes that are built to do that, but if you are already writing the software at a low level, then why not write the OS to do some direct addressing across the whole data center – including through the network switches. This will save an immense number of cpu cycles.
5. Compress the Data: Data compression saves space, but usually at the expense of a few cpu cycles, so there’s a trade-off to calculate here. Depending on circumstances you might be able to build a really impressive compression capability. There might even be the possibility of using solid state storage rather than hard disk. Solid state uses less electricity. The whole equation here may be very complex and is highly application dependent.
6. Data Architecture. You can go with any of the free databases (MySQL, PostgreSQL, Firebird, etc.) But you may be inclined to use your own data structures. All the commonly used databases, whether free or paid for, were built yet again for the average circumstance. But you have the opportunity to build a data service for a specific circumstance. Remember for example that Google uses Map Reduce for some of its data. If all the data is going to be held in addressable solid state storage then any existing database is definitely the wrong product and will be horribly inefficient, because it is assuming the data comes from a spinning disk.An in memory structure will be far more efficient.
7. Back-up, Disaster Recovery, Follow the Sun: The 3 data center strategy with mirroring (which I spoke of in The Death of the Data Center (Part 4 – Power Distribution and Cooling)) looks promising as a means of never needing to have back-ups (or UPS). It looks really inexpensive for disaster recovery and if you have a global business you may even be able to load balance across 3 data centers in the US , Europe and the Far East. In many instances the workload will follow the sun.
8. System Management: There’s a potentially big win in the system management area. Part of root cause analysis and maintenance can be carried out preemptively by service management processors. Because you control the OS you can also insert an agent in every instance of the OS – an agent that gathers the data you need (and no other data). You will end up with a purpose built Configuration Management Database (CMDB) that actually works. You can ensure purity of software across the data center and you can also choose to upgrade the whole environment only every 18 montsh, say.
9. Security: You can bolt the whole data center down with a set of closely controlled permissions. The main worry is that someone (external or internal) manages to run a rogue process that does something nefarious. But if there’s only one closely controlled mechanism for loading any executable, that’s never going to happen. You can even design that in to the OS and the system management capabilities. Security is always an afterthought in corporate IT, but it’s actually possible to write it into the application.
10. Client caching: Put as much processing as possible on the customer’s client device (PC or MAC or even Smartphone) so that it doesn’t get executed in the data center. Naturally you will put the interface on the client, but you can certainly design the architecture so that it makes maximum use of the client. The beauty of this is that the customer probably won’t mind because most client devices have cpu cycles to spare.

I have not compiled this list with the idea that all scaled out data centers will, in the future, adopt  all these technical tactics. All I’m demonstrating here is that there are many areas of attack, where the designers of a data center can reduce the number of cpu cycles required to carry out recurring tasks. The motivation to do this will be very high. In the last 20 years we’ve watched cpus grow increasingly powerful and seen their power squandered by programmers that no longer cared to write efficient applications.

Well, the motivation for efficiency has now returned.

Also:

The Death of the Data Center: The Model
The Death of the Data Center: Location, Location, Location
The Death of the Data Center: Power
The Death of the Data Center: Cooling
The Death of the Data Center: Networking
The Death of the Data Center: Server Hardware
The Death of the Data Center: The Software
The Death of the Data Center: Software Optimization

In my rough model of scaled-up data center costs, I never had a separate category for software costs. These were simply bundled in with the server costs. You can think of this as a natural consequence of the costs of an operating system normally being bundled in with the server when you buy hardware.

Note that I was also modeling a highly scaled operation. If it’s Infrastructure as a Service (IaaS) then all you are going to have is operational software; the applications come from the customer. The same will be true of Platform as a Service (PaaS), it’s just that you will also be providing a software stack and development environment. With Software as a Service (SaaS) you’ll be providing the application and in most circumstances that’s going to be an application your company built.

Open Source Software

In the last decade Open Source software has become part of the fabric of the IT industry. It is  available for a wide variety of tasks. Some of it is very high quality and nearly all of it can be used for no license fee, as long as you obey the associated license. Open Source software has already become a business factor in the ISP business with most ISPs providing a easily installed and highly functional software stack for building web sites. It is going to play a similar role in the aaS businesses, but not just because the cost is very low.

A critical factor is going to be that the source is open.

As an operator of a large data center you will not have much of a problem paying for support from any given Open Source team, but you probably wont want support of the normal kind. In all probability you will change the source code to tailor the software you use to precisely fit your environment.

Just think of the OS for the moment. Under normal operation an OS will have many background processes running. All such processes have a function and quite a few of them run by default, whether you have a need for them or not. Some of them are keeping logs, some are handling messages from the network, some are there to fire off scheduled jobs if there are any, some handle printing, some provide directory services and so on. They all sit there happily chewing up cpu cycles whether they are busy or idle. You don’t want any of them there unless they have a specific role to play.

In a normal environment you would never even think of deleting these useful background processes, but in an environment that cares about efficient resource usage you really don’t want superfluous anything. Not only that, you maybe interested in rewriting some of these tasks; because you need them, but in your brave new world you may need them to run slightly differently. That’s why open source is key.

The Context Change

You may not know what a context change is. Think of it like this. You are getting on with some task; writing a program or writing a report or designing something and you get interrupted by a phone call. It’s someone on the line and they need your attention at once. You give them the assistance they need and five minutes later you’re back to what you were doing.

The question is: how long will it take for you to get back to the level of productivity you had before the phone rang?

It varies from person to person, but the general consensus is somewhere between 5 and 10 minutes. Computers are the same. They don’t like context switches either. With a computer the context switch occurs when it has to drop one application to get on with another. Technically, this is simply because the computer has to put a kind of bookmark in the application it is running, save it in a state where it can start up again and then load another application – filling the instruction pipeline to the processor. Context changes are expensive, both for people and computers. They consume cycles.

All those background jobs that do printing or write logs or handle network traffic cause context switches. If you can eliminate some of this activity you win more than you might think. And if you know precisely what your workload is going to be then you can design the whole flow of instructions and data to the cpu to try to minimize context switches.

All I’m really trying to do here is point to the fact that, although we have had general purpose computers for decades, we now have the possibility of building very application specific scaled  computers.

The big point here is this:

Executing cpu instructions is what a data center does.

That’s it’s prime job. That’s why it is there. If you improve the efficiency of that activity then you win EVERYWHERE:

• Less electricity
• Less cooling
• Less hardware
• Less networking

In the next posting I’ll cover optimization. We’re not done here yet.

The Death of the Data Center: The Model
The Death of the Data Center: Location, Location, Location
The Death of the Data Center: Power
The Death of the Data Center: Cooling
The Death of the Data Center: Networking
The Death of the Data Center: Server Hardware
The Death of the Data Center: The Software
The Death of the Data Center: Software Optimization

Just as Apple was unable to keep a lid on the news of its iPhone before its official announcement, it hasn’t been able to completely quell the speculation surrounding its imminent media tablet. In its teleconference to discuss its most recent and very impressive quarterly results Apple’s COO, Tim Cook, was derogatory about netbooks yet again. Apple is clearly positioning itself to produce “the device that the netbook wants to be, but clearly isn’t.”

I’ve just reviewed 10 Pointers To What Apple’s Netbook Will Be Like and right now I can’t disagree with anything suggested there. It seems pretty much on the mark. The new device will be a tablet, no keyboard laptop and the price point will fill the gap between the iPhone and the low end Apple laptop.

Will Apple cannibalize its laptop Market?

This is a question that can’t be answered precisely without spending a fair amount of time using the still-a-matter-of-speculation device.  The truth is that the netbook has cannibalized the laptop market in a big way. There are two kinds of buyer in the netbook market; the price sensitive buyer and the laptop buyer.

1. The price sensitive buyer: The price sensitive buyer is buying a device that wasn’t previously available at such a price point. To some buyers, devices in the $300 price area are “throw away” purchases. So what if I have to throw it away? Some buyers are buying because the price point is now low enough for a device for the kids. In developing countries, these buyers are buying because the price is now low enough for them to own a computer.  The point about these buyers is that they are all new buyers. They are not buying the netbook instead of some other device.
2. The laptop buyer: There is one huge advantage that the netbook has over every other variety of laptop. It doesn’t weigh very much. Weight is a big buying criteria. Netbooks have already spread themselves out across a wide price range to reflect this fact. In general you use fewer applications on a laptop, so you need less computer power, unless the laptop is also used as a desktop. Synchronizing information between laptops and desktops in now a long-solved problem. When a laptop buyer buys a netbook, a laptop sale fails to happen.

That’s the problem that Apple faces. There is little doubt that it will do an excellent job with its media tablet. It will have very little difficulty positioning its device as “the device the netbook wants to be” and it will be instantly popular on the day that it’s released. But it will have some difficulty preventing the cannibalization of its own laptop market.

A Media Device is a Media Device

A partial solution to the cannibalization problem is to make the new media tablet more like the iPhone than the MacBook. Have all the applications fed to the new device from the Apple App Store and have the interface be very touch oriented. Apple will need the consumer to believe that: if I need a keyboard, I need a laptop, if I don’t I need the tablet. That won’t prevent cannibalization, but it will keep it to a minimum.

The other part of the game will be to make this the media device to die for. The goal will be that it becomes the must have video device or games device for an airline journey. Again this is more like the iPhone than a laptop. And Apple will surely include the possibility of it being able to be a phone, complete with carrier contract.

I said most of what needed to be said about Amazon’s Kindle in the posting
One Million Users: Is Stanza Killing The Kindle? I’ll reiterate some of what I wrote then, but I’ll add to it. There are 2 points:

1) The Kindle is, by design and by definition, a niche device.

Amazon may have had the idea that it could launch a compelling pervasive substitute for a book, but it was wrong. It might have the best possible display and the best possible interface for reading a book, but if you already have an iPhone and/or a netbook, why do you want to carry a Kindle around with you. It’s a device too far. You’d have to be an aficionado of some kind to want a Kindle and the aficionado market is not big enough to get the volume.

The Kindle never started life as a niche device, it just got pushed into a niche by the iPhone and it was going to get further isolated as the netbook market produced better devices. So Amazon is now in a difficult position in one respect. The Kindle isn’t going to sell in the tens of millions. Not now. Not ever. Whether the true market for the Kindle is big enough for Amazon to want to keep it alive is difficult to know. That’s all about the numbers.

It could sell a much larger volume if it sold the Kindle from book stores, but which book store would ever carry the device? That’s part of the problem. None of the natural retail outlets for the Kindle will touch it.

2) The big danger to Amazon was that Stanza (or some other program) would become a real Amazon competitor.

In a few months Stanza, the dominant iPhone book reading app, achieved a million downloads. That must have made Jeff Bezos go white. A relatively simple piece of software dashed past the Kindle and opened up the prospect of a competitor to Amazon in the electronic books market. It had already rolled up an inventory of 100,000 books.

To its credit, Amazon struck back as fast as it could. It introduced its own Kindle app for the iPhone by the beginning of March, in effect admitting that the Kindle was now trapped in a niche. I’ve no idea when Amazon opened up negotiations with Lexcycle (the Stanza company) but I suspect it watched the downloads of its Kindle app for a few weeks and then made the phone call. Better to buy Lexcycle out before it got too big.

Is The Kindle Toast?

So Amazon is now the gorilla in the iPhone book reading market. Good for Amazon. Nicely executed. Amazon is not going to have to face some energetic little start-up that might give it a run for its money. Take out handkerchief. Wipe brow. Phew!

Or maybe not. What if Apple is about to launch its not-a-netbook-netbook, a media pad that plays video and games and music and is a low end lap top. Why wouldn’t Apple set up it’s own electronic book store. It could. If it did the Kindle would be toast and Amazon would be challenged.

In my rough model of scaled-up data center costs, the highest cost was server hardware, making up 36.9% of the total, including the cost of data storage. You don’t need to think hard before you realize that such costs can vary dramatically. Consider data storage costs. If your data center is feeding video streams to the Internet from a vast video library, like YouTube does, the storage requirement is huge compared to streaming SMS length messages to the world, as Twitter does. Indeed Twitter doesn’t even store its billions of tweets indefinitely, so it can estimate its storage requirements easily on a per user basis, whereas the YouTube library just keeps on growing forever.

The same is true of servers or server boards. You have to have sufficient cpu power and memory for the primary workload. So life is a great deal easier if you have a single workload rather than any kind of mixed load. This in turn means that the economies of scale are going to be better for Software as a Service (SaaS), than for Platform as a Service (PaaS), which in turn will be better than Infrastructure as a Service (IaaS). It’s really all about workloads.

Server Economies of Scale

Let us not forget that we are talking about massive scale here. When you are an old-style data center buying new hardware, you may be able to go to HP and IBM and demand a significant discount on the hardware you intend to buy over the next few years. If you are building a scaled out data center that is designed precisely for a specific workload, you are more likely to go to HP or IBM to discuss the design of the hardware you are going to buy. In fact you might not go to any of the traditional hardware vendors. Instead, you might go to an engineering design company that will design the boards and the networking switches for you and then take the contract to a manufacturer in Taiwan to build the precise hardware that you want.

The point is that if you are going to buy thousands or tens of thousands of servers, you’ve moved to a level of demand where you have choices that the typical data center does not have. You are like a manufacturer designing a new plant who needs to make precise decisions about the tooling of the plant, right down to the design of the robot welders.

I’m not being critical of the server products that are built and delivered by the big computer manufacturers. Such engineering is difficult to be critical of, but all such servers, whether mainframes or cheap commodity server boards are designed for “the average circumstance.” It’s really unlikely that your requirements are anywhere close to the average.

Think first about cooling. Servers and blade cabinets are generally built to go in glass room data centers with raised floors and atmospheric cooling. Cooling is one of your major costs, so you will want the server boards and the data storage to be designed to be exactly complementary to the cooling system and you’ll probably want much more focused cooling.

Now think about cpu, memory and disk. You will want a specific ratio between these that fits the workload. There is no point in having surplus anything, because surplus memory or cpu or disk needs cooling if it’s turned on, and if it isn’t turned on, why have you got it? Now think about whether you want local disk or a huge SAN, or a combination of the two. Think also about failover and redundancy and how the server hardware blends with the networking hardware.

Here’s the point:

If you don’t get the hardware right, then you made a mistake with the biggest cost element in your whole operation.

Only a rank amateur would make a mistake like that.

See also:

The Death of the Data Center: The Model
The Death of the Data Center: Location, Location, Location
The Death of the Data Center: Power
The Death of the Data Center: Cooling
The Death of the Data Center: Networking
The Death of the Data Center: Server Hardware
The Death of the Data Center: The Software
The Death of the Data Center: Software Optimization