Chapter 1: The Big Picture

Back to Table of Contents.

What Is Enterprise?

You may have heard that big Internet sites like or eBay or Google have thousands— sometimes tens of thousands, or more— of servers powering their websites. If you’re reading this book, you’ve probably already built at least one web application of your own, and it probably had only a handful of machines behind it, perhaps even just one application server and one database. In fact, maybe you had shared hosting and only had a fraction of a full server at your disposal.

If you had a great idea for an online business and were given 1,000 servers, what would you do with them? How would you make the most of them? What operational goals would you define for reliability and speed, and how would you leverage all of that hardware to achieve those goals?

Before diving into the pieces of an enterprise system, or discussing how to build one, a good starting point is to simply define enterprise.

Unfortunately, that is not an easy task. There is no particular set of tools that, if used, will make your architecture qualify as enterprise, even if the word “enterprise” is in the product names of the tools you use. The big companies listed above have built many of their own tools to support their software stack, but they are definitely still “enterprise.” Similarly, there is no single configuration of pieces that together spell enterprise. If you looked at the configuration of Google’s servers and compared their configuration to Amazon’s, the two would look quite different. But they are both enterprise nonetheless—it just happens that the two enterprises have different goals, and therefore need different architectures to reach those goals.

In some sense, a site is enterprise when it feels like it is. All of the Internet behemoths once started out with a single application server and single database, just like you. It’s anyone’s best guess when they crossed the blurry line into “enterprise.”

That said, there are certainly some criteria that, when satisfied, do make a site feel like it’s enterprise. These criteria are topics of this book, and will be referred to again and again:

  1. It’s fast. You can define a service level agreement (SLA) for how long it takes each component to do its job, which in turn allows you to define an SLA for end-to-end load times of any given web page.
  2. 2. It is always available. You can define an SLA for your minimum uptimes for all critical components, and you aim for “four nines”—99.99% uptime.
  3. It scales linearly. You can scale to hundreds of thousands or even millions of users by adding additional hardware.
  4. It is fault-tolerant. If noncritical components go down, the majority of functionality stays intact, and your users don’t know the difference.

There are other criteria that make your site feel like it’s enterprise, too, although they are mainly operational concerns, and aren’t covered in depth this book:

  1. All source code is in a source control repository.
  2. All new code goes through a QA cycle before it is deployed.
  3. There is a deployment procedure, and failed deployments can be rolled back.
  4. Errors are logged in a central location, and the appropriate personnel are notified in real-time.
  5. Log files and databases are backed up in a central location.
  6. Statistics about the website’s operation can be collected and analyzed to determine which areas need attention.

Implicit in the above list are a number of job functions and departments other than software development. Reading between the lines, you find:

  1. A database administrator (DBA) who sets up failover databases and ensures backups are available. A DBA can also tune database configuration parameters and control the physical mapping of data to disks to improve performance. Many also consult on schema design to ensure optimal performance and data integrity.
  2. A quality assurance engineer (QA) who tests release candidates before they are put into production and tracks issues to be fixed by developers.
  3. An operations or release engineer who manages the releases, creates deployment plans, and rolls out your new software in the wee hours of the night.
  4. An information technology engineer (IT) who maintains internal machines that house backups, log files, etc.

Having these people in your organization will push your systems architecture toward “enterprise.” Similarly, designing your system to be enterprise creates the need for all of these individuals. In some sense, when your company itself feels like an enterprise, your software is probably getting to be enterprise, too. When the two are out of step, you will know it, because either half of the engineers have nothing to do, or else everyone is stepping on each other’s toes.

Growing Slowly

Every website begins its life with a single developer and a single line of code. Figure 1-1 shows a simple configuration of a Rails application connected to a database. You will likely spend quite a bit of time developing your application on this setup before it’s ready for its first user.

er_0101Figure 1-1. A basic website configuration

When it’s time to launch, there some issues that ought to be considered. Figure 1-2 shows the same configuration as above, but with redundancy at the application level, and failover at the database level.

There are two copies of the application so that in the event one machine fails, there is still another that can handle incoming traffic. Similarly, in the event of a hardware failure on the database machine, a copy that is a transaction or two behind can be brought online quickly.

er_0102Figure 1-2. A basic website configuration with failover and redundancy

Even if you are barely using any of the resources on either the application or database machine, redundancy and failover are a very good idea. At this point, neither of these considerations is aimed at managing load—that comes later. Rather, both are intended to ensure the availability of your web application. Reprovisioning a machine, configuring it, and loading all your software and data from backups can cause quite a bit of downtime. During that time, your customers will can find your competitors’ sites, and they are likely to form negative opinions about your site’s reliability as well.

With this configuration, and perhaps even a good deployment strategy, there is plenty of work within the application and data layers that can be done before you need to add any additional complexity to your system in the form of encapsulated services or asynchronous processes. Depending on the feature set of your web application, this may even be as far as you need to go. You are already satisfying a number of the criteria that define the elusive concept of “enterprise.”

There is, within an enterprise, the need to scale horizontally as well. Only so many engineers can work in one codebase before no one can work at all. Even if there is only one chef in the kitchen, there is still only so much space for the sous-chefs.

A common way to deal with this human scaling problem is to break up a large application into smaller pieces, or services, each responsible for a specific function within the enterprise. It’s no surprise that the software splits often follow organization boundaries, so that individual teams can take on full ownership of their pieces of the overall system.

Each service has its own full stack that mirrors the stack of the traditional website from Figure 1-2. The difference is that a service is responsible for a small fraction of the duties that make up the entire website, usually one specific, specialized group of related functionality. It’s possible—and sometimes preferable—to abstract all database access behind services. The front-end website then becomes a consumer of these services and has no need for a database of its own, as shown in Figure 1-3.

er_0103Figure 1-3. A front-end website backed by services, which in turn are backed by relational databases

When you add services into the mix, it’s hard to argue your system is not enterprise.

There are a number of other components commonly found in an enterprise setup. Figure 1-4 shows a generic enterprise configuration. Powering the front-end website are a number of services. There are also a collection of asynchronous processes that receive information from services via a messaging queue. In addition to the front-end website, there is a web services layer aimed at providing external clients with a subset of the functionality available inside the firewall. There is also redundancy and failover in all critical places. Finally, each service database feeds a data warehouse, which powers site reporting and decision support.

Note, of course, that simply replicating this configuration is not enough. Each piece of the system is an independent, isolated, and encapsulated system in its own right, and deserves thorough and thoughtful design. What goes where, and how to implement each individual unit is as much an art as it is a science.

er_0104Figure 1-4. A generic enterprise architecture with redundancy and failover

Understanding All the Pieces

This section gives a brief introduction to each piece of the enterprise system.

Persistence Layer

The persistence layer is where you store your business’s data. As the name implies, data here sticks around for a long time; it persists until you explicitly change it or remove it. Most frequently, the persistence layer is a Relational Database Management System (RDBMS).

Because protecting your data is critical, the persistence layer should provide certain guarantees, collectively referred to as ACID: atomicity, consistency, isolation, and durability. Each of these properties plays a different role in maintaining the integrity of your data:


The ability to group a number of operations together into a single transaction: either they all succeed, or they all fail. The RDBMS should ensure that a failure midway through the transaction does not leave the data in an intermediary, invalid state. For example, a bank account transfer requires debiting funds from one account and crediting funds in another. If one of the operations fails, the other should be rolled back as well; otherwise one account may be debited without making the corresponding credit in the other account.

Consider the following instructions:

# power failure happens here

If the database fails between the two statements, where we have a comment to the same effect, the user of the ATM system will likely see an error on-screen and expect no transaction took place. When the database comes back up, though, the bank customer would be short $50 in account one, and be none the richer in account two. Atomicity provides the ability to group statements together into single, atomic units. In Rails, this is accomplished by invoking the method transaction on a model class. The transaction method accepts a block to be executed as a single, atomic unit.

Account.transaction do
  # power failure happens here

Now, if the power goes out where the comment suggests, the database will ignore the first statement when it boots back up. For all intents and purposes, the first statement in the transaction never occurred.


The guarantee that constraints you define for your data are satisfied before and after each transaction. Different RDBMS systems may have different allowances for inconsistency within a transaction. For example, a complex set of bank transfers may, if executed in the wrong order, allow an account to drop to a balance below zero. However, if by the end of all the transfers, all balances are positive, the consistency check that all balances are positive has been guaranteed.


The guarantee that while a transaction is in process, no other transaction can access its intermediary—and possibly inconsistent—data. For example, a bank deposit requires checking an account’s existing balance, adding the deposit amount to this balance, and then updating the account record with the new balance. If you are transferring $100 from one account to another, with one statement to debit $100 from the first account, and another statement to add $100 to the second account, isolation prevents your total balance from ever appearing to be $100 less between the two statements. Figure 1-5 illustrates this. Without the transaction in thread 1, the output time 3 would have been 0 + 100.

er_0105Figure 1-5. Transactions isolate multiple queries into a single atomic unit


The guarantee that once your database accepts data and declares your transaction was successful, the data you inserted or modified will persist in the database until you explicitly remove it or modify it again. Similarly, data you deleted will be gone forever. There is no code example to demonstrate durability. It is a property of how the RDBMS interacts with the operating system and hardware. Short of a disk failure that actually destroys data, if the database returns control to you after a statement, you can assume the effects of the statement are permanently stored on disk, and a reset of the database or some other activity that clears system memory will not affect this assumption.

Note that many databases do allow you to relax the durability restriction to increase speed at the expense of reliability, but doing so is not generally recommended, unless your data is not very important to you.

Of course, having a database that is ACID-compliant is not enough to guarantee your data’s integrity. Armed with this set of guarantees, it is now up to you, the database designer, to set the database schema up properly to do so. In Chapters 4 through Chapter 9, we will build up a complex schema, and then provably guarantee our data’s integrity.

Application Layer

The application layer represents everything that operates between the data layer and the outside world. This layer is responsible for interpreting user requests, acting on them, and returning results, usually in the form of web pages. When you start your first Rails project by invoking rails {projectname}, what you have created is the application layer.

Depending on what your application is, the application layer can have different relationships with the data layer. If, for example, the purpose of your website is to provide Flash games for visitors to play, the application layer, and most developer effort, will focus on the games users play. However, the application layer may also facilitate user login, as well as storage and retrieval of high scores in the database.

More commonly, though, websites present information to users and allow them to act upon it in some way. For example, an online news site that displays articles, or a movie ticket vendor site that provides movie synopsis and show times in theatres nationwide. In these cases, the application layer is the interface into the data stored in database.

In its simplest form, a single Rails application comprises the whole of the application layer. When a user requests a web page, an instance of the full application handles the request. The entry point into the code-base is determined by the requested URL, which translates into a controller class and action method pair. Code executes, usually retrieving data from the database, culminating in the rendering of a web page. At this point, the handling of the request is complete. This is the simplest of architectures, and was shown in Figure 1-1.

A front-end and services

The configuration above can take you quite far, but it can only take you so far.

As your company grows, the complexity of your business needs may become too large to be managed well within a single application. The complexity can come either in the user-interface, or in the back-end logic that powers it.

If you operate a blog site that is wildly successful, you may want a variety of different user interfaces and feature sets based on the target audience. For example, the young-adult site may have a feature set geared toward building discussion within a social network, while the adult-targeted site might be devoid of the social aspect altogether, and instead include spell-checking and other tools to make the content appear more polished. Or you might even spin off a completely different application that uses the same underlying content structure, but with a completely different business model. For example, For example,a website for submitting writing assignments, where students are able to read and comment on other students’ work, could easily share the same underlying data structures. Figure 1-6 illustrates.

er_0106Figure 1-6. An application layer split into a single service and many front-ends

The first front-end may be the teen-targeted site, the second the adult-targeted site, and the third the homework-submission site. All three contain only the user-interface and the workflow logic. They communicate over the network with a single content management service, which is responsible for storing and retrieving the actual content from the database and providing the correct content to each site.

The opposite type of complexity is perhaps even more common. As your website grows in leaps and bounds, with each new feature requiring as much code as the originally launched website in its entirety, it often becomes beneficial to split the application up into smaller, more manageable pieces. Each major piece of functionality and the corresponding portions of the data layer are carved out into its own service, which publishes a specific API for its specialized feature set. The front-end, then, consumes these service APIs, and weaves a user interface around them. Based on the level of complexity and the need to manage that complexity, services can even consume the APIs of other services, as well.

In this configuration, shown in Figure 1-7, the front-end is a very manageable amount of code, unconcerned with the complex implementations of the services it consumes behind the scenes. Each service is also a manageable piece of code as well, unconcerned with the inner workings of other services or even the front-end itself.

er_0107Figure 1-7. An application layer split into a single a single front-end, and many back-end services

Web-services layer

Services, as in service-oriented architecture, and web-services are distinct, but oft-confused concepts. The former variety live within your firewall (described later in this chapter), and are building blocks of your larger application. The latter, web-services, straddle the firewall, and provide third-parties access to your services. One way to think of this distinction is that the services have been placed “on the public web.” Functionally, a service and a web-service may be equivalent. Or, the web-service may impose usage restrictions, require authentication or encryption, and so on. Figure 1-8 shows a web-service backed by the same two services as the front-end HTML-based web application. Users equipped with a web browser visit the front-end HTML site, while third-party developers can integrate their own applications with the web-service.

er_0108Figure 1-8. A web-service backed by service applications

This was the briefest introduction to SOA and web-services possible. In Chapter 13, we’ll look much more closely at what a service is, as well as how one fits into a service-oriented architecture. We’ll also examine a variety of circumstances to see when moving to SOA makes sense – their organizationally or technically. In Chapter 14, we’ll go over best practices for creating your own services and service APIs, and in Chapters 16 and 17, we’ll build a service-oriented application using XML-RPC. In Chapter 19, we’ll build a web-service using REST.

Caching Layer

All of your data lives in databases in its most up-to-date, accurate form. However, there are two shortcomings to retrieving a piece of data from the database every time you need it.

First, it’s hard to scale databases linearly with traffic. What does this mean? Imagine your database system and application can comfortably support 10 concurrent users’ requests, as in Figure 1-8.

er_0109Figure 1-9. An application capable of supporting 10 user requests

Now imagine the number of requests doubles. If your application adheres to the share-nothing principal encouraged in Dave Thomas’s Agile Web Development with Rails, you can easily add another application server and load balance the traffic. However, you cannot simply add another database, because the database itself is still a shared resource. In Figure 1-9, the database is no better off than it would have been with 20 connections to the same application server. The database still must deal with 20 requests per second.

er_0110Figure 1-10. Double the requests

The second shortcoming with requesting data from the database each time you need it is due to the fact that the format of information in the database does not always exactly match the format of data your application needs; sometimes a transformation or two is required to get data from a fully-normalized format into the objects your application can work with directly.

This doesn’t mean there’s a problem with your database design. The format you chose for the database might very well be the format you need to preserve your data’s integrity, which is extremely important. Unfortunately, it may be costly to transform the data from the format in the database to the format your application wants it in each time you need it. This is where caching layers comes in.

There are many different types of, and uses for, caches. Some, like disk caches and query plan caches, require little or no effort on your part before you can take advantage of them. Others you need to implement yourself. These fall into two categories: pre-built and real-time.

For data that changes infrequently or is published on a schedule, a pre-built cache is simple to create, and can reduce database load dramatically. Every night, or on whatever schedule you define, all of the data to be cached is read from the database, transformed into a format that is immediately consumable by your application, and written into a scalable, redundant caching system. This can be a Memcache cluster or a Berkley Database (BDB) file that is pushed directly onto the web servers for fastest access (Figure 1-10).

er_0111Figure 1-11. BDB file caching system

Real-time caches fall into three main categories. The first and simplest real-time cache is a physical model cache. In its simplest incarnation, this is simply an in-memory copy of the results of select queries, which are cleared whenever the data is updated or deleted. When you need a piece of data, you check the cache first before making a database request. If the data isn’t in the cache, you get it from the database, and then store the value in the cache for next time. There is a Rails plug-in for simple model caching called cached_model, but often you will have to implement caching logic yourself to get the most out of it.

The next type of real-time cache is a logical model cache. While you may get quite a bit of mileage out of a physical model cache, if your application objects are complex, it may still take quite a bit of processing to construct your objects from the smaller constituent pieces, whether those smaller pieces are in a physical model cache or not. Caching your logical models post-processed from physical models can give your application a huge performance boost. However, knowing when to expire logical model caches can become tricky, as they are often made up large numbers of records originating in several different database tables. A real-time logical model cache is essentially the same as a pre-built cache, but with the added complexity of expiry. For maximum benefit, a physical model cache should sit underneath and feed the logical model cache rebuilding process.

Note that both physical and logical model caches must be shared in some way between all application servers. Because the data in the cache can be invalidated due to actions on any individual application server, there either needs to be a single shared cache, or otherwise the individual caches on each application server need to notify each other of the expiry somehow. The most common way this is implemented in Rails is via Memcache, with the configuration shown in Figure 1-11.

er_0112Figure 1-12. Memcache configuration

The final type of real-time cache is a local, user-level cache. In most load-balanced setups, it’s possible to have “sticky sessions,” which guarantee that a visitor to your site will have all of her requests handled by the same server. With this in mind, and with some understanding of user behavior, you can preload information that’s likely to be useful to the visitor’s next request and store it in a local in-memory cache on the web server where it will be needed. This can be information about the user herself, such as her name and any required authentication information. If the last request was a search, it could be the next few pages of search results. Figure 1-12 shows a user-level cache, local to each application server, backed by Memcache.

er_0113Figure 1-13. User-level cache configuration

Depending on the nature of your application, as well as where your bottlenecks are, you may find you need one type of cache, or you may find you need all of them.

Messaging System

A messaging system, such as an Enterprise Service Bus (ESB), allows independent pieces of your system to operate asynchronously. An event may occur on your front-end website, or perhaps even in the service layer, requiring actions to be taken that do not affect the results to be presented to the user during their current request. For example, placing an order on a website requires storing a record of the transaction in the database and processing credit card validation while the user is still online. But it also may require additional actions, such as emailing a confirmation note or queuing up the order in the fulfillment center’s systems. These additional actions take time, so rather than making the user wait for the processing in each system to complete, your application can send a single message into the ESB: “User X purchased Y for $Z.” Any external process that needs this information can listen for the message, and operate accordingly when it sees a message it’s interested in.

Using a messaging system like this, which allows many subscribers to listen for any given message, allows you to add additional layers of processing to data without having to update your main application. This cuts down on the amount of systems that need to be retested when you add additional functionality. It also cuts down on the number of issues the application responsible for user flow needs to be aware of. That application can concentrate on its specific function—user flow—and other parts of your application can take over and manage other tasks asynchronously.

Web Server

The web server has a relatively simple role in the context of a Rails application. The purpose of the server is simply to accept connections from clients and pass them along to the application.

The one subtle point to keep in mind is that users visiting your site with their web browsers are not the only clients of your application who will interact with the web server. When you break up your application into separate services with separate front-ends, each piece is a full Rails application stack, including the web server.


The firewall is the barrier of trust. More than protecting against malicious hacking attempts, the firewall should be an integral part of large systems design. Within the firewall, you can simplify application logic by assuming access comes from a trusted source. Except for the most sensitive applications (e.g., controlled government systems that must protect access to secret information) you can eliminate authentication between different pieces of your application.

On the other hand, any piece of the system that accepts requests from outside of the firewall (your application layer, front-end, or web services) may need to authenticate the client to ensure that client has the correct level of access to make the request.

Preface Chapter 2 : Organizing with Plugins