Databases > Using Databases

Online databases - a dream too far?

By Rory Ridley Duff, First Contact Software Ltd

This article by Rory Ridley-Duff first appeared in Computanews issue 109 in October 2000. It was updated by Rory in June 2002 to take account of recent technology developments. The debate about web-based database applications and what they can realistically deliver is still very much a live issue despite these advances in Internet technologies. For an alternative view, read Matt Harrison's response to the original article.

It is difficult not to be excited by the Internet, and in particular the emerging technologies that enable web-pages to retrieve data from on-line databases. Surely the day is not far away when we can do away with Microsoft Access, Visual Basic and Delphi, and write our database applications to run in Internet Explorer? No more difficult installations! No more processor and memory hungry applications demanding that we upgrade all our workstations! Just point your browser at a location on your intranet or the Internet and away you go.

A Future Full of Potential

Those of us who increasingly use the Internet will all know sites that show the potential of e-databases. I will never forget buying my first book at www.amazon.com. I browsed their 5 million-entry catalogue, read the book reviews posted by previous buyers, made my choice, paid my money, and received my book the next day. This is the stuff of the future - I need never enter a bookshop again!

The number of Internet sites that interact with on-line databases is growing rapidly. Even in the not-for-profit sector, there are sites that show the potential of web-enabled database applications. The Royal National Institute for Deaf People (RNID) have a website that enables visitors to search for organisations that provide help or specialised products for people who are hard-of-hearing. The data is input and managed using their database application. A web-developer has then created an interface that enables visitors to retrieve contact details and information by entering relevant keywords. How far away is the day that fully functional database applications (such as contact management or accounting software) will run from Internet Explorer?

Sadly, the truth is that this type of application is still a long-way off, or may never be practical to develop. The Internet is not designed for database application development, and while it has a role in making information stored in databases widely available to millions of people, it is likely (certainly for the next 3-5 years) to be largely restricted to simple database applications.

A Future Full of Danger

Computer Weekly published research in November 1999 forecasting that investment in Internet applications will rise dramatically through 2001/2, and then fall away sharply in 2003/4. The research anticipates that about 75% of Internet development projects will be abandoned (yes, 3 out of every 4) and that it will not be until 2005 that the technologies (and companies) will emerge that will improve the success-rate and viability of large-scale Internet application development. You should, therefore, think carefully before investing in the development of Internet database applications.

Technology Limitations

There are good reasons to be cautious. Firstly, the Internet is designed to enable as many people as possible to use it and this limits its usefulness for certain types of database application. Database specialist Jim Falino has written enthusiastically about Active Server Pages - the Microsoft technology for database querying web-pages - but he also makes clear its limits:

…every request for something from the server must establish a new connection. This means that if your Web page has three GIF files and two Java applets, then five separate, consecutive connections must be made and broken. While that's an awful lot of overhead (and would make one lousy database application), the fact that you don't have a permanent connection to the server allows your Web site to be "hit" by an almost unlimited user base -- like the world!

Most database applications maintain an open connection to a server to ensure that data can be read, written and processed as quickly and efficiently as possible. A web page, however, must issue a request each time it wishes to retrieve data, or write it. This may be adequate for on-line book ordering, or one-way information retrieval, but is not suitable for a telephone helpline or a sales department, which requires near instant access to caller details (including previous enquiries) and the ability to quickly input call details, information requests, and to send responses.

The second reason is that web pages are written in Hypertext Markup Language (HTML). HTML is essentially a document formatting language. Put simply, it describes how to display textual and graphic information on a page. Fine, we can use it to design nice looking pages to display information. After gaining considerable experience and expertise we can even create forms that people can use to input information which we can send to a database. While there are new ways to store structured data, the output is still rendered in HTML so its limitations are still relevant. The shape of the future is looking clearer. The W3C group have formalised standards for structured data (XML) and also how to transform it (XSLT) and format it (XML-FO).

Unlike HTML, which is a language aimed at page layout, we can now read or embed structured data, such as a directory of contacts or products, and use a language dedicated to selecting and presenting it in a variety of formats. At the same time, Microsoft are now heavily promoting .NET, a framework for application development that fully supports the XML standard, and also makes deployment of applications on both the desktop and the Internet much easier. Their Internet Explorer browser technology is also ahead of the competition and is the only one to support all the current W3C recommendations.

It looks set to become a de facto industry standard, particularly for database applications on the Internet. Taking the next step and developing a fully functional database application is rather more challenging - rather like writing it in Microsoft Word using Visual Basic for Applications. It can be done, but is costly, difficult, and it places strict limits on what it is possible to do well. Producing outputs in HTML, or rendering XML in HTML format, still feels like going back to the 1980s where the developer has to write a stream of text into a file and control its formatting in order to produce a report. While we have more tools to work with legacy systems, we are still a world away from the graphical form and report designers that we have become used to using.

So What is Possible?

At First Contact Software Ltd, we are concerned that organisations wishing to develop Internet applications are realistic in their expectations. Many enquirers about our First Point of Contact database software ask us when it will be web-enabled. This is a difficult question to answer with precision, because the short answer is somewhere between already and never. It is already web-enabled in the sense that its data (which can be in an Access database or SQL Server database) can already be queried and displayed on web-pages; no change to the database or its software is required. The tools required to do this are readily available and we have already undertaken projects to integrate the contact data in First Point with Internet search tools so that people can search an online directory.

First Contact Software LTD recently won a Smart Award for innovation in design. This has enabled us to fund research and development into technologies that can support on-line information and trading systems. We have also been able to consult on the human issues (data quality, data management, data ownership) that affect implementation in leading non-profit and commercial organisations. What has emerged from this work is that the Internet is a good collection and presentation tool, but insecure for mission-critical and highly complex business operations.

The data management issues are substantial. Who is responsible - legally and administratively - for ensuring published data is accurate? How can this be effectively managed by the publisher of the data? It is clear that effective and secure systems will never run completely in a web-browser because security and data management issues are best addressed through in-house procedures. Once the application and data are out on the web, control can be lost.

At present, a web-hosted database application will still perform poorly where there is a large amount of data management. Microsoft's .NET significantly improves form design capabilities for browsers and is closing the gap with traditional development tools and approaches. However, the performance issues regarding client and server side operations remain, and applications hosted entirely on the Internet will still not perform as well as systems where functionality (and data) is held in the location that best serves local as well as central needs.

Secure communication is also a key issue. Any open connections to a database make a whole web-site vulnerable to attack from hackers. It is necessary to use a secure messaging technology that allows packages of data to be sent and received at regular intervals. We have identified a product called Zero-Click from a company called WWM. This can create packages of encrypted data so that it is possible to design applications that send or receive records to either local or remote computers without opening up a security loophole. So long as the application does not need to be in real-time (with up-to-the-minute rather than up-to-the-hour information) this architecture can deliver secure applications that perform well. The architecture radically affects system design and will require database applications to be conceived and constructed in a different way.

Terminal Services

Internet application development as the best way to provide remote access to central data has a new competitor. With Windows 2000 Server, Microsoft started bundling Terminal Services. This is a technology that brings back the benefits of the mainframe world where client workstations receive screen updates but processing taking place on a central computer. Terminal Services allows organisations to run all their applications centrally, while still giving users their familiar Windows environment.

With so much invested in Windows skills, this is an attractive option that allow organisations to continue using their old applications and development skills, but avoid all the installation problems associated with ageing workstations and old operating systems. The main case for Internet application development is low deployment costs. Terminal Services has taken away the need for Internet application development if the only need is access to existing systems or data from remote sites. As an alternative, it competes well where the number of users is between 5 and 50, but as the number rises costs escalate and Internet application development is again a competitive solution.

When using Terminal Services the Internet is at most a communication tool (clients can connect through the Internet to an office-based server). This can bring down communication costs - particularly now flat-rate access to the Internet is becoming affordable. It make systems for small multi-site companies cost-effective when compared with the full Internet approach.

Conclusions

The Internet and the world wide web are maturing rapidly and offer huge potential for a multitude of applications. They cannot, however, escape their underlying design limitations, which allow as many users as possible to 'hit' web-sites and retrieve essentially static text and graphic pages of information. This limits their usefulness for database applications that require instant access, dynamic updating of information and fast processing. In addition to (relatively) slow communications, the page design language of the web still limits database form design.

While technologies such as Cold Fusion, Active Server Pages and Microsoft's .NET are maturing and making new applications possible, the best approach is likely to be a combination of client interfaces (running either in a browser or on the desktop) with server-side components (that handle requests for data). Internet applications that query databases are now common projects, but they remain expensive to develop while software companies build components that can be reused on future projects. The future looks brighter: the cost of web-developers has fallen dramatically following the collapse of many Internet projects and companies; development tools are also in their second generation and are improving all the time. While costs are falling, they will remain substantially higher than other forms of software development for many years.