Cost Benefit Analysis of Micro Data Center Deployment for Edge Computing

Several IT trends including internet of things (IoT) and content distribution networks (CDN) are driving the need to reduce telecommunications latency and bandwidth costs. Distributing “micro” data centers closer to the points of utilization reduces the latency and costs from the cloud or other remote data centers.

This article provides a cost benefit analysis of micro data centers compared to traditional enterprise data centers and shares why micro data centers are best suited to support edge computing.

This article examines why micro data centers are best suited to support edge computing over other alternatives such as server rooms and traditional data centers.

Read more in this whitepaper, which covers a cost benefit analysis of micro data centers compared to traditional enterprise data centers.

You’ll learn:

• Drivers that favor the adoption of micro centers.
• IT technologies that make single-rack micro data centers possible.
• The capital cost advantages of micro data centers compared to centralized data centers.
• The future for micro data center architecture.

Content Summary

Executive summary
Introduction
Drivers of micro data centers
IT enablers
Capital cost advantages
A future micro data center architecture
Methods of deployment
Conclusion

Executive summary

Several IT trends including internet of things (IoT) and content distribution networks (CDN) are driving the need to reduce telecommunications latency and bandwidth costs. Distributing “micro” data centers closer to the points of utilization reduces the latency and costs from the cloud or other remote data centers. This distributed data center architecture also provides physical infrastructure benefits that apply to any small data center regardless of the latency requirement. This paper explains how micro data centers take advantage of existing infrastructure and demonstrates how this architecture reduces capital expenses by 42% over a traditional build. Other benefits are discussed including shorter project timelines.

Introduction

The increasing consumption of digital services through our mobile devices, our homes, our offices, and countless other applications (i.e. manufacturing, mining, oil & gas, etc.) is changing the way data centers are deployed. As more and more demand is placed on data centers for these services, data takes longer to reach its destination. In general, the further the distance from the data center, the longer it will take to deliver its digital services and with increased bandwidth costs. Placing data centers closer to the point of use is known as edge computing. An example of this is EdgeConneX, a company that deploys colocation data centers on the edge of the network to reduce latency and bandwidth costs for their customers.

This paper establishes why micro data centers are best suited to support edge computing over other alternatives such as server rooms and traditional data centers. First we explain the drivers that favor the adoption of micro data centers, then we discuss the IT technologies that make single-rack micro data centers possible, and then we highlight the capital cost advantages of micro data centers (physical infrastructure only) compared to centralized data centers. Finally, we propose a future micro data center architecture that could replace traditional enterprise data centers for certain applications. Though not the subject of this paper, micro data centers are also effective solutions for branch office environments.

In this article we define a single-rack micro data center (Figure 1) as a self-contained, secure computing environment that includes all the storage, processing, and networking necessary to run applications. They ship in a single enclosure and include all necessary power, cooling, security, and associated management tools (DCIM). Micro data centers are assembled and tested in a factory environment.

Drivers of micro data centers

“Drivers” motivate or prompt some action. There are four main drivers of micro data center adoption to support edge computing applications, over alternatives such as server rooms and traditional data centers. We discuss each of these drivers in the following subsections.

• Scalability
• Speed of deployment
• Reliability
• Outsourcing to the cloud and colocation

Scalability

Scalability is a strategy to conserve capital by paying only what you need, when you need it. A lesson learned from the “dot com” bubble of the early 2000’s when oversized data centers were built for an expected IT load that never materialized. Micro data centers can more easily be “stepped and repeated” to accommodate growth in IT gear as the need for more compute arises. Once fully utilized, another one is deployed in the same facility or in a different facility depending on the available electrical, space, and bandwidth capacity. Their standardized, prefabricated nature along with smaller kW increments is fundamentally what makes them a highly scalable solution compared to traditional “stick-built”, purpose-built data centers.

Speed of deployment

Some businesses encounter situations where they need to deploy a computing solution quickly. Since micro data centers are a complete physical infrastructure solution (including the IT gear and, sometimes, the software), it becomes feasible to significantly decrease a project timeline compared to a traditional data center or server room. This is because micro data centers eliminate the need to design, specify, procure, and integrate a group of disparate components. The speed with which you can deploy a micro data center is strongly dependent on how standardized it is. The more standardized the micro data center, the more likely it is to be a stocked item. However, as the kW capacity of a micro data center increases, the less likely it is to be a stocked item given the higher carrying cost. Larger IT deployments (e.g. 1MW) still require purpose-built or colocation data centers. None the less, a micro data center will always be faster to deploy compared to a traditional data center or server room. In some cases, configuration tools (Figure 2) are available to quickly and easily customize standard solutions by allowing users to change power, security, and IT equipment to an enclosure. A rules-based configuration engine should check for incompatibilities between components and, when complete, an order can be created.

Reliability

In general, the reliability of a data center is undermined by the extent to which its critical systems are customized. This is why a standard-model vehicle, is more reliable and less costly than a custom-built, one-off vehicle. While larger data centers could be standardized, it’s much easier and practical to standardize smaller ones. This is a key reliability advantage of micro data centers.

One of the ways designers increase the reliability of traditional “one-off” data centers is by adding redundant systems. When geographically distributed, such as with edge computing, data centers also gain reliability improvement through geographic redundancy with the use of automated workload mirroring and orchestration. This is analogous to distributing the storage drives of a RAID array across the country and using software to recover data if any of the drives fail. In terms of data centers, the virtual machines, and their data, are preserved in much the same way. The smaller the data centers get, the more distributed, less oversized, and resilient this array of data centers becomes. This is another key reliability advantage of micro data centers.

Outsourcing to the cloud and colocation

CIOs who decide to move their IT gear from on-premises data centers to collocation facilities, or who decide to outsource applications to cloud providers, are still left with a certain amount of IT gear in corporate and branch offices. Micro data centers are ideal in these cases to support this IT gear because of their small footprint and ability to be placed in an open office area. This allows the existing data center or server room space to be converted to office space, or leased out. For example, the cabinet in Figure 3 could be mistaken as office furniture but is actually a micro data center in an office environment.

IT enablers

“Enablers” are things that make it possible or easier to accomplish something. There are three main information technologies that enable the use of micro data centers. The drivers to deploy micro data centers, described above, are independent of IT enablers. We briefly explain these technologies and describe how they simplify and or reduce the cost of micro data center deployments.

• Hyperconverged IT
• Compaction
• Virtualization

Hyperconverged IT

If micro data centers were deployed with disparate IT parts (i.e. servers, storage arrays, network switches, management software, interconnecting cables, etc.), the integration of all these pieces would limit the “speed of deployment” advantage of micro data centers.

At a basic level, hyperconverged IT refers to the integration of compute (i.e. processor), storage (i.e. hard disk drive), and networking resources into a single package (i.e. rack, chassis). When all of these parts are integrated as a single solution, you improve the speed of deployment for the entire micro data center. In addition, IT reliability improves and, to a certain extent, the amount of physical space occupied is reduced because the converged stack is designed as a single solution from the ground up.

Compaction

Although there is some degree of compaction in relation to hyperconverged IT, the compaction we refer to here is the result of technological improvements in IT equipment. For example, the continuous reduction in transistor size has decreased the size of the chips in IT equipment while simultaneously increasing performance. Data storage has also seen significant compaction, especially with the advent of solid state drives. All of this compaction in footprint, with simultaneous improvement in performance, has made it possible for a single rack of servers to process the same IT workload as 13 racks of servers 8 years ago.

Virtualization

While compaction makes it possible to consolidate 13 racks of gear into a single rack, virtualization is what allows administrators to harness all of that compute power across various workloads. Virtualization can refer to server, storage, and networking and it is this abstraction of all three that allow geographically distributed micro data centers to appear as a single physical data center.

Capital cost advantages

The biggest capital expense advantage that micro data centers have over building a traditional centralized data center, is that they can typically run off of a building’s existing physical infrastructure. In many cases, existing buildings have spare power capacity to support a micro data center both from utility and emergency generator power. In other words, micro data centers can utilize the “sunk costs” in facility power (e.g. switchgear), cooling (e.g. chillers), and core and shell construction, making it less capital intensive than building a new data center.

There are also some tax benefits associated with micro data centers. Since they are typically considered “business equipment” as opposed to a “building improvement”, they can be depreciated over a shorter life. White Paper 115, Accounting and Tax Benefits of Modular, Portable Data Center Infrastructure, provides more details on this subject.

Analysis methodology

We performed a capital cost analysis of a data center architecture deployed with micro data centers, and compared it to a traditional centralized data center. For this analysis, we used the same data center capital expense model as that used by the Data Center Capital Cost Calculator TradeOff Tool shown in Figure 4. The tool estimates the capital cost of materials, labor, and design for each subsystem in a data center. For the centralized traditional data center, we used the data center cost value shown in the results section of the tool. We compared this traditional data center to 200 micro data centers (single rack 5 kW/rack) distributed throughout various buildings. Note that this doesn’t mean you would need 200 buildings since more than one micro data center could be housed in a single building.

The micro data center capital cost was estimated based on micro data centers available on the market today. For the micro data center, we eliminate the capital cost (materials, labor, and design) of the subsystems that are typically present in buildings (sunk cost). These subsystems included core & shell, raised floor, switchgear, and generator.

Assumptions

Table 1 provides the pertinent assumptions used in the capital cost analysis. The “Environmental and security” cost component includes cameras, temperature, humidity, and smoke sensors. The inputs shown in Figure 4 provide other details for what is included in the traditional capital cost.

This analysis also assumes that the applications running in a traditional data center can be partitioned across multiple micro data centers interconnected by a wide area network (WAN). This “logical” data center is enabled by hyperconverged infrastructure and low latency links between the individual micro data centers. There are some applications that may not be compatible with this type of architecture. We discuss the possibility of this type of architecture in the next section.

Findings

The capital expense for building a single centralized data center rated for 1MW of IT load is $6.98 million or $6.98/watt. The capital expense of 200 5kW micro data centers is $4.05 million or $4.05/watt. Micro data centers represent a 42% savings over a centralized data center. Note that if fire suppression was removed from traditional and micro data centers the overall savings would increase to 48% savings. This is because the pre-action sprinkler system cost in the traditional data center is significantly less than the NOVEC clean agent system assumed in the micro data center.

The UPS in the micro data center has more total UPS capacity than the traditional data center. When building a single 1 MW data center, the UPS is typically oversized by 1.2 times the IT capacity. However, when the UPS is distributed across 200 racks, a 1.2 oversizing factor results in a 6 kW UPS, which may not be enough. Because of the load diversity effect, the UPS capacity in a distributed architecture should be slightly higher to accommodate occasional rack densities above the 5 kW/rack average. In this analysis we used an 8 kW UPS. The diversity effect is one of the drawbacks of distributed architectures in general. Despite this drawback, distributed architectures bring an advantage over traditional data centers in that they are highly scalable.

Micro data centers are inherently scalable because one can deploy them in their entirety only when needed. In the case of this analysis I can deploy them 5 kW at a time. This “pay as you grow” approach conserves cash flow and is especially useful in branch office type of deployments.

A future micro data center architecture

The introduction of hyperconverged infrastructure and its management software has made the concept of a software defined data center (SDDC) easier to implement. In essence SDDC brings some of the benefits enjoyed by internet giants to enterprise companies. Micro data centers take the SDDC concept a step further by distributing the compute, storage, and networking resources across different geographic locations, as opposed to placing all the IT gear in a single centralized data center.

This concept of managing workloads across multiple data centers is not new and is used by a few cloud service providers including Amazon and Google. However, the idea of applying SDDC to distributed micro data centers and managing them as a single logical data center is new for enterprise companies that don’t have the scale of internet giants. An example of this type of distributed computing is Qarnot Computing. This company processes information on distributed micro data centers and uses the waste heat of the micro data centers to heat residential homes.

This data center architecture has the potential to save enterprises significant capital expenses as demonstrated in the analysis above. Another benefit is lower operating expenses in terms of specialized IT admins dedicated to storage, networking, and compute. Hyperconvered infrastructure abstracts these three resources into a single hardware stack doing away with specialized hardware equipment for each resource. All the distributed micro data centers act as a single data center that provide IT services similar to a utility.

One major unknown, and risk, for this future data center architecture is the effect of network latency on the distributed micro data centers. Google uses dedicated “non-public” fiber links between its data centers to allow data flow between them. This is cost prohibitive for typical companies, therefore, the links between micro data centers will need to be made using “public” WAN connections and with enough bandwidth for the applications they support.

Methods of deployment

There are three basic ways you can deploy small data centers:

1. Build it from scratch by specifying, buying, and assembling individual parts
2. Build it with a preconfigured solution and assemble the parts
3. Build it using a micro data center which comes prefabricated and fully assembled

In moving from the first method to the last method you encounter more standardization and fewer choices in the design of the data center. Users who require custom color schemes, special electrical outlet configurations, specific cooling system, etc. may need to use method number one. Furthermore, you have the choice of buying the entire solution from a single vendor or separate parts from multiple vendors. Buying the entire solution from a single source means that you need a bid from each vendor (a minimum of two bids required), while buying 6 separate parts (e.g. rack, power, cooling, storage, compute, and networking) from separate vendors means at least 12 bids required. Then all the parts must be integrated which may cost extra if you use an integrator.

The trend in the IT industry is toward standardized hyperconverged IT infrastructure that is easy to deploy and operate. If you plan on using a hyperconverged IT solution, we believe it’s logical to deploy your physical infrastructure using micro data centers. The intention of this method is to deploy the entire solution, including the IT compute, storage, and networking gear, as a single module. This method increases the predictability of the installation, operation, and overall reliability of the data center.

Conclusion

The benefits of distributing micro data centers are that they scale as needed, reduce computing latency, and reduce the risk of bringing down the entire data center operation (i.e. reduce single points of failure). Similar to a distributed IT architecture, if more capacity is needed in the future, another micro data center is added. Standardizing these micro data centers results in further benefits including reduced deployment time, simplified management, and lower maintenance and capital costs.

Source: APC