Oded Noy had faced challenges before -- when he co-founded an application management startup, for instance, or when he participated in war game simulations for the Israeli Air Force. But this was unique: Create a scalable platform that would transform the online car shopping and financing experience for consumers -- in six months.
It was September 2005, not long after Noy, as CTO of a new startup called Zag, had led the development of a Java platform for distributing inquiries and leads to auto dealers. So at least he wasn't starting from scratch.
But Capital One, a major provider of auto loans, had just partnered with Zag to fill a much taller order: Give consumers no-hassle pricing and availability for new and used cars, as well as the ability to configure new vehicles and arrange financing online. That required bringing a whole new set of applications into Zag to handle the inventory management, configuration, pricing, call center, and CRM. The technology Noy's team developed would underlie a new Capital One Web site, DriveOne.com, but it also had to be flexible enough to serve future partners that might have other requirements.
For Noy, the answer lay in SOA (service-oriented architecture). Its abstracted, modular approach is fundamentally about designing for extensibility and adaptation to new business needs. "It lets you build a framework that can incorporate unknowns, providing a cohesive platform even when developing rapidly -- without later requiring a re-architecting," Noy says.
But SOA or not, Zag couldn't develop all the needed applications in time. So the company acquired Autoland, a credit-union auto-buying service. Autoland's technology included call center and CRM applications for the automobile industry, as well as back-end transaction handling. Zag's own technology was based on Java, whereas Autoland's technology was largely based on COM (Common Object Model) and Visual Basic, with a smaller set of .Net and Microsoft SQL Server components, as well.
In other words, Zag suddenly had a legacy to deal with -- it made no sense to buy Autoland's technology and then reinvent it in Java. Plus, there were some functions, such as rapid prototyping for SQL Server and managing thumbnail images, that were more easily done in .Net. The architectural vision came into focus: Use the best technology for each job, keep specific modules on one technology base for simplicity, and use an SOA approach to pull it all together.
Speeding development with composite apps
Zag needed to develop, deploy, and integrate 10 core applications in all, most of which drew upon the functionality of existing Zag and Autoland apps. To achieve that goal on time, "we needed a composite application environment very early in this effort," recalls Nick Pernblad, Zag's chief software architect. And that meant defining services that would form the building blocks for the applications.
A key principle behind SOA is to isolate the building blocks of business processes and develop services that instantiate them and then combine those services as needed using well-defined interfaces. Services are typically distinct functions that work in multiple business process, such as "look up dealer address" or "validate sign-on information."
Many companies see the economies of scale afforded by reusing such services as the main benefit of SOA. But Zag's most immediate need was very rapid app dev. And self-contained services reduced the need to rework applications continually as services were changed or added.
"We do a lot of things that aren't terribly difficult, like display inventory or car options. But as you put it together, boy, is it a big application," Noy says. The composite-app approach keeps each task manageable. It also allows Zag to "carve out" the needed functions from the apps acquired from Autoland rather than rewrite them or subsume the entire system. "You get a rapid development environment that is as rapid as the business needs," he says.
The composite application approach requires ensuring that services don't duplicate efforts, which can be difficult when software must interact with external partners that may define their own functionality differently. It's also critical that the interfaces be thoroughly thought out and implemented well. "Things break at the interfaces, but well-defined interfaces let you mitigate the risks," Noy notes. "A clear definition can be hard to define," he adds. For example, it's clear that a dealer address is part of the dealer module, but it's not so clear whether price information resides in the pricing system or the inventory system -- especially if some dealers treat inventory as a function of their pricing strategy.
Distributing the architecture
The 10 composite applications that make up the Zag system are deployed on one of two platforms -- .Net or Java -- so internal interactions within a composite app are native. "This approach permits higher performance and simplifies development," says Yoshi Okayama, Zag's vice president of software development. Some composite applications have dedicated databases and/or servers. For example, the configuration and pricing composite app has its own database for vehicle information. To the other composite applications, the particular deployment configuration is hidden behind a well-defined interface.
Enterprises often use an orchestration mechanism such as an ESB (enterprise service bus) or object broker to manage the service calls. But Zag didn't have the time to develop or deploy such an infrastructure if it were to meet its six-month deadline. So Noy's team took a decentralized approach to delivering services and managing interactions within and among composite applications.
Zag's composite apps integrate through point-to-point connections, relying on Path Reliability's application monitors and strictly defined process calls to ensure that services aren't combined in unplanned ways. This approach demonstrates that you don't need to build an extensive infrastructure before you can take advantage of SOA, Noy notes. Instead, you can stage the degree of infrastructure as needed over time.
The point-to-point communication mechanisms vary depending on the circumstances. For performance reasons, in intra-app communications, native .Net or Java binary protocols were often used. For most interfaces between Java and .Net applications, Zag's development team opted for the basic Web services protocols SOAP and WSDL.
In other cases, it made more sense to use a REST (Representational State Transfer) approach, where method call parameters are passed as a URL and the returned data is a lean XML document. "This works particularly well with AJAX [Asynchronous JavaScript and XML] clients, making the Web pages essentially a composite app," Pernblad notes. For example, the inventory application is a completely self-contained AJAX app that knows how to search and render inventory data. The JavaScript libraries can be dropped into any type of Web page, whether Active Server Pages or JavaServer Pages.
Pushing performance
Another benefit of this AJAX approach is that it off-loads processing to the client. "You can do more on the front end with AJAX and today's clients," Noy says. "The classic Web services approach would [have the client page] just call the remote service and do XSL [Extensible Stylesheet Language] translation. But who said there's only one way to draw these lines?"
Zag's Web pages are part of the composite application matrix in that they are assembled at the edge of the network by an Akamai sever. This allows the system to cache page fragments that change less frequently, while permitting other, more dynamic page fragments to interact independently with several composite application services. The services remain common across all customers, while the assembled HTML reflects the needs of a particular customer.
A major component of the auto-buying service involves frequently updating the price and configuration database and joining it with the dealer inventory database so that users see only relevant information for their area. The standard SOA approach is to have a service extract the data from each database and then generate the appropriate XML.
"For performance reasons, that won't work in a two-hour window," Noy notes, given the many large feeds involved. So, Zag simply has the two databases directly perform a join each day. "If you are doing data-intensive things, do it where the data is," he says. Zag's composite apps then access the joined database -- in which the data is already tagged and sorted to reflect business requirements -- to respond to user queries. "We exposed a well-defined interface of stored procedures to do this task quickly," Noy adds.
Blazing a new trail
Zag's current model for integration relies on direct connections between services, rather than the message brokering offered by ESBs or conventional EAI software. But as the complexity of interactions among services increases, Noy expects a more centralized orchestration approach will be needed. As an ESB infrastructure is phased in, the existing services will plug in to the bus, allowing for central monitoring and management, as well as taking advantage of security and protocol translation services provided by the ESB.
Zag's experience demonstrates that there is no single correct approach to SOA. The architecture can vary while still delivering on the key SOA principles of functional abstraction, component modularity, and business-process-aligned services. "As long as the interfaces are well-defined and the responsibilities are clearly delineated, how you arrange the pieces should be up to the project's needs, not some preordained model," Noy says.
