The E-Business Software Weekly is a series profiling trends and developments in software and applications that support e-business, the Internet, and other electronic communication channels. Look for a new story each week in this space.
Application Integration: A Primer
EAI, RPC, ORB, SOAP, JMS, CBD, J2EE. The list reads like an alphabet soup of government agencies. In reality, it represents perhaps the most important factor in the future of the Internet and e-business.
That factor is "application integration," or the seamless connection of disparate software applications so that they perform, in time and functionality, as if they were components of the same application. Historically, sophisticated software applications have operated as standalone entities, performing tasks and generating results within their own particular domains, and then depositing this information into large, sequestered databases. Sometimes, other software applications could be made to query or transmit data into these isolated databases, but any kind of systematic connection between two or multiple applications has generally required large, complex software integration projects sometimes lasting years and costing into the millions of dollars.
These complexities have restricted application integration projects to only the largest corporations with the largest bank accounts and the most sprawling organizations, in which the integration of corporate processes becomes an operational necessity. The vast majority of corporations, by contrast, have had to make do with only limited data and application integration supplemented by the largely manual transport of data among applications and databases on an as-needed basis. While the inconveniences and inaccuracies introduced by these non-integrated systems have made most corporations less operationally efficient than otherwise possible, these organizations have usually come to accept such shortcomings as the unavoidable costs of doing business in a complex, multifaceted world.
Even the arrival of the Internet did not, at first, seem to alter this equation. After all, the Internet on the surface is nothing more than a massive collection of isolated and unconnected Web sites. To be sure, as e-commerce has matured, many Web sites have been tied into back-end ordering, inventory, and accounting systems. But there has been little interconnection among Web sites themselves, and even less among Web-based applications. As a result, the Internet has tended to operate as an extension of traditional business operations (e.g., a new channel) rather than as a distinct business environment. In other words, while the Internet has dramatically extended the reach (in time and space) of traditional business operations, it has had a much less significant impact on the way in which these operations have been conducted.
The Role of Application Integration
And so, application integration has played a limited role in the development and functionality of the online world to date. How, in that case, can one assert that it is a key to the future of the Internet and e-business?
The answer begins with the value proposition of e-business itself. The promised benefits of e-business-reduced operating costs, enhanced service, and greater market reach-have ignited an explosion of e-business software applications designed to serve a wide variety of needs and industries. Completely new functions, like electronic customer relationship management (eCRM), customer self-service, electronic payments, and vendor-managed inventory, have entered the business vocabulary thanks to the new capabilities offered by e-business. And customers themselves have modified their buying habits even as their expectations for customer service have increased, and with customers now routinely demanding such services as online order-tracking and real-time inventory status. These developments, taken together, have dramatically heightened the competition among both traditional and Internet-based businesses, creating a "digital arms race" in which competing firms continually attempt to leapfrog one another by exploiting the Internet in order to deliver even better customer service while making their operations more efficient than ever before.
And that is where application integration comes in. "Achieving these improvements via the Internet," says a recent report by the Boston-based research consultancy Aberdeen Group, "requires companies to integrate data and content among legacy, packaged, and custom applications" on a more or less ongoing, company-wide basis. They note, for example, that:
In order to offer superior customer self-service regarding order or account status, companies must create a unified view of the information maintained by separate customer-facing and back-office business applications.
In order to handle a purchase requisition submitted to a manufacturer, a customer-facing procurement application needs to communicate in real-time with product catalogs and Enterprise Resource Planning (ERP) systems, enabling it to respond to a customer's questions about specifications, options, availability, and price.
In order to lower costs and delivery time among supply chain partners, participating companies must integrate applications so as to provide for collaborative planning and forecasting, coordinated manufacturing, and coordinated distribution management.
As vital as application integration is in these and many other respects, however, it does not come without some serious challenges. Firms undertaking integration projects must complete their work at a rapid pace in order to keep up not just with the emergence of new e-business applications, but also with their equally aggressive competitors. Yet, as Aberdeen notes, studies show that application integration projects have generally produced "higher costs, longer projects, and higher levels of complexity than customers anticipated." Some research, in fact, indicates that as many as two-thirds of integration projects either are never completed, or else exceed their budgets and timetables by a substantial degree.
Thinking Strategically
So is application integration really worth all the trouble? The fact is, for enterprises wishing to compete on the Internet-and, increasingly, even in the offline world-there really isn't any choice. If applications cannot work together, most of the benefits of e-business disappear, or at least are greatly diminished. Those companies that do manage to integrated their applications therefore will achieve a powerful competitive advantage, allowing them, over time, to secure increasing market share through a combination of improved service and reduced costs. And so, for most companies, the question really becomes, not whether to integrate applications, but how?
"In a Web-enabled world," the Aberdeen researchers explain, "all IT investments must be viewed as strategic." That is, IT projects must be seen more than mere technical projects isolated not only from one another but from a company's overall business mission. Rather, these IT projects create the foundation upon which e-businesses operate, and so must be viewed-and undertaken-in a more global perspective, akin to a strategic business plan or strategic marketing plan.
Call it a "strategic technology plan."
Pursuing IT projects in this strategic vein requires that both business and technology executives think, more carefully than ever before, about the role of individual IT projects in the company's overall technology framework-and that technology objectives be defined within the context of business objectives, rather than independently of them. It also demands that responsibility for developing and implementing these projects be sufficiently unified so that this company-wide perspective can be attained. Aberdeen puts this mandate bluntly: "Managing the diverse software components in the e-business infrastructure requires an integrated view and a single point of control."
Good technology alone will not bring these conditions about. As the Aberdeen researchers note, "a popular misconception is that middleware technologies solve all of the really difficult integration problems. While such technologies help a great deal, many of the problems to be solved are not technology-related-e.g., problem definition, integration planning, identification of requisite skills, and correct staffing."
A Practical Approach to Integration
What is needed, say the researchers, is a systematic application management process to both supplement and give direction to the underlying technology solutions. In a November 2001 white paper entitled, "E-Business Infrastructure Integration: Practical Approaches," Aberdeen sets forth some of the key components of this process:
Integration requirements analysis and migration planning. Traditionally, companies have defined integration projects based primarily on the isolated functionality sought within a single integration. Now, with e-business requirements evolving so rapidly, greater attention must be paid from the outset to the deployment of a technical infrastructure that can not just satisfy current requirements, but that can easily integrate with and migrate to new and emerging technologies.
Technology selection. Investing the time to pick the right technology and supplier, the researchers insist, is more important than it has ever been. Aberdeen defines a variety of criteria that should be applied in selecting technologies, including extensibility, reusability, interoperability, ease of maintenance, and long-term cost-effectiveness. In addition, they say, selected suppliers should possess an in-depth knowledge of both the applications to be integrated and of the requirements and potential pitfalls involved in such an integration.
Organizational preparation. Integration projects, the Aberdeen team says, must be managed differently than traditional software development projects because there are often unprecedented dependencies on both other projects within the company and on external resources. Integration solutions also must be designed from the start to work with other applications' data models, APIs (application programming interfaces), and architectures. Based on its research, Aberdeen believes that, in the optimally executed integration project, some 40% to 50% of the overall project budget and resources should be devoted to analyzing the integration requirements and designing an appropriate solution.
Integration management. Integration project managers, note the researchers, "need to have a solid understanding of all aspects of the project. The project requires far more management, coordination, and communication than a stovepipe application development project." Moreover, because integration projects are intended primarily to solve business problems, the management team should extend beyond IT specialists to include business analysts, business process owners, subject matter experts, and end users "who can apply their areas of expertise to minimize the risk of making incorrect tradeoffs and inadequate technology choices."
Partnering decisions. Historically, most application integration projects have been undertaken in-house, but companies are increasingly turning to outsourcing as a more effective-and more cost-effective-approach to integrating applications. For example, a recent CIO Insight survey of 400 chief information officers and senior IT strategists revealed that 76% outsourced application development and IT services. When these respondents were asked which specific functions they outsourced, 30% listed application integration. While selecting an outsourcer can be even more consequential, and hence more demanding, than selecting technologies and suppliers, Aberdeen notes that the right outsourcer often can significantly reduce both costs and time-to-deployment while generating a greater return on investment than would be the case with in-house development.
Integration Best Practices
Having the appropriate application integration process in place is the first step toward a successful integration-but only the first step. Aberdeen also advises companies venturing into this potential minefield to learn from the sometimes difficult and costly experiences of their predecessors. To promote this understanding, Aberdeen has identified a number of best practices that have helped both to reduce application integration costs and to maximize the chances of project success:
Establish business goals to be achieved by the integration project. These goals will serve as a compass to keep projects from drifting into technical development areas that either bear little relevance to the core business goals or that fail to deliver on key business needs. Such business goals should be supplemented by specific metrics that can be employed, on an ongoing basis, to assess adherence to the goals.
Actively participate with end users. The executives, business analysts, marketing personnel, and sales professionals who will eventually be using the integrated applications can provide invaluable insight into front-line needs and application usage patterns that can help define which aspects of an integration project are likely to deliver the greatest benefits, and which are apt to be superfluous. End-user participation at the definition stage also can create a sense of ownership and excitement on their part that can help make the integration deployment more successful while smoothing over any rough spots.
User a single, flexible, and consistent process framework. Because application integration initiatives typically involve the integration of many disparate application modules, integration can be extremely resource intensive. "Using a single, flexible, and consistent framework for each process," say the researchers, "greatly reduces the amount of personnel needed to develop and maintain the system" by minimizing the time required to reconcile inconsistencies among individual modules' architectures, data, and computer code.
Make data profiling and mapping a key part of the project setup. Understanding the actual content and structure of legacy data sources based on the physical data, and not on obsolete documentation, say the researchers, "has proven to significantly reduce project failures and prevent schedule overruns." Defining and correlating different data sources ahead of time, the researchers note, "will go a long way toward providing consistent information across the enterprise."
Build application integrations that are scalable and extensible. The only certainty in an integration project is that, over time, it will grow larger and expand to incorporate other applications and business processes. In the mainframe era, new applications were often patchworked into existing systems, creating an application framework that grew more incomprehensible as each new application was added. On the other hand, by adequately planning for scalability and extensibility, application integration can become easier rather than more difficult as time goes on.
Maximizing Integration Benefits
The field of application integration is a technically complex and administratively challenging one. But the business benefits of an application integration well-pursued can be substantial. Aberdeen concludes, based on its review of previous integration projects, that following guidelines like those presented above can reduce integration development time and costs by between 30% and 50%, improve programmer productivity by 30%, improve application performance by 20%, and trim maintenance costs by 30% to 50%-all tangible business results that will be registered in corporate balance sheets for some time to come.
The bottom line, say the Aberdeen researchers, is that companies need to understand as completely as possible all the facets of the integration landscape before they set to work. "It is imperative," they contend, "that the integration team does its homework to fully capitalize on the benefits of the technologies, tools, integration methodologies, and services available in the market," and hence on the long-term business benefits that well-executed integration projects can provide.
More E-Business
Software Weekly stories
[an error occurred while processing this directive]
ADVERTISING FEATURE
