Introduction

The Eclipse Data Tools Platform (DTP) project is a new top-level project at eclipse.org. Originally proposed by Sybase in February, 2005, DTP has attracted strong community support and is currently managed by a committee comprised of Sybase, IBM and Actuate. Since DTP is in the early milestone phase, this is an opportune time to explain the motivation and current plans of DTP to the Eclipse community at large. Doing so will provide valuable context to interested parties – both potential consumers and contributors – and enable additional community involvement at this crucial formative stage.

Motivation for DTP

In a very short time span Eclipse has become a premier development platform, and a vibrant ecosystem has grown around it. Built for extensibility from the start, Eclipse has enabled and nurtured extensions by providing solid technical, educational, and mindshare leadership in the development platform space. Indeed, it no longer is a question of whether Eclipse will succeed, but rather the extent to which it will prosper. A key element in this growth is the vibrant extender community.
    
Eclipse has always been easy to extend at the micro-level. Using the plug-in infrastructure, developers contribute functionality to the base platform that seamlessly integrates with existing capabilities. In fact, a well-designed plug-in is almost indistinguishable from the base platform itself. At a macro-level, a number of successful commercial products have been built on Eclipse. These products often take advantage of the “re-branding” features of Eclipse to replace the splash screen and welcome pages, and generally give that installation of Eclipse a personality keyed to the particular product. Finally, an important class positioned between these levels has appeared.

In between the micro- and macro-level, a number of common functional areas have emerged. Often these mid-level contributions encompass much more than would sensibly fit into one (or several) plug-ins, but are not sharpened in focus to a product level. Two important areas include enabling technologies, often surfaced as frameworks, and exemplary uses of these frameworks, often surfaced as tools. Examples of such mid-level initiatives include the Eclipse Web Tools Platform (WTP), the Eclipse Test Performance Tools Platform (TPTP), and the Eclipse Business Intelligence and Reporting Tools (BIRT) projects. Such initiatives effectively become platforms, comprising multi micro-level contributions, upon which additional contributions, both at the micro- and macro-level, can then be constructed.

The motivation for platforms typically is to aggregate and provide consistency over a certain problem domain. Often, the necessity of the platform has been shown by multiple, conflicting contributions at the micro-level. The community comes to realize that everyone would be better off by building a commons, and a platform project is proposed. Crucially, for a platform project to succeed, deep and broad community support is required.

DTP arose from such a need. Clearly the use of data is a foundation of computer applications, and components facilitating data use are highly desirable. Shortly after Eclipse was introduced, a number of micro-level contributions in this problem domain appeared, and the number has grown significantly over time. Unfortunately, so has fragmentation among these tools. While a respect for diversity is a key part of encouraging innovation, fragmentation of enabling technology only serves to hamper progress. In brief, we’d like to concentrate on higher level problems; we’d like to make the challenges of developing software with Eclipse be based in the problem domain, not in the plumbing. We’d also like to use tools each bringing new and useful functionality, and avoiding overlap. Personally, I want to use a multiplicity of tools without having to repeatedly define connections to data sources and hunt around for locations of data access drivers. DTP intends to meet this need in the data frameworks and tooling area.

Scope of DTP

Since data is foundational in computer applications, the scope of DTP can be difficult to define. Clearly we’d like to capture the common intuition that certain frameworks and tools naturally belong in DTP, whereas others do not. As a first approximation, assumed by many when first encountering DTP, the scope is limited to database frameworks and tools. Specifically, this assumption concentrates on relational database systems, and looks for tools such as SQL editors and result set views. Clearly, components for relational database systems naturally falls within the scope of DTP, but limiting consideration to this area is not the intention of DTP. Notice that DTP is the Data Tools Platform project, not the Database Tools Platform project.
    
So how do we broaden the scope of DTP without including too much? The DTP community discussed this issue at length during the proposal phase, and a lot of effort went into defining the “Scope” section of the DTP Charter (http://www.eclipse.org/datatools/dtp_charter.html). We will take some time to explain this section in detail, since it is vital that the Eclipse community at large understand the intended scope of DTP. When the scope is understood, the consumers will know what to expect from DTP and future directions of possible development. Potential contributors will understand if DTP is the right top-level project to approach at eclipse.org.
    
Many options were considered to define the scope of DTP. Lists of examples – both in current plans and projected in the future – provided a starting point. Such lists, however, seemed to say both too much and too little. They said too little in the sense that the relevance of areas not mentioned was hard to determine. Also, they said too much by indicating future potential directions are more likely than others. Looser definitions were attempted by coining words and phrases, but this was undesirable because it seemed artificial and not truly capturing the intuition shared by the DTP community that there are some sensible boundaries for the scope. Finally, after many attempts, we settled on defining the problem domain that DTP intends to address, and then placing that domain within containment relationships in set of projects hosted by eclipse.org.
    
The problem domain that DTP addresses is for data-centric frameworks and tools. Here, the idea of ‘data-centric’ means those applications having an association with a data source. Obviously, that casts a very broad net, and perhaps it is even difficult to imagine applications not having an association with a data source. The next move is crucial, however, to setting the scope boundaries. The data-centric problem domain is seen as the base, and other problem domains specialize it. For instance, Java source code is data, and tools that work on this data – such as a Java source code editor – are data-centric. The question then becomes whether a more specific domain for this data can be found within other eclipse.org projects. In the case of Java source code and a Java source code editor, the answer clearly is ‘yes,’ since the problem domain of “Java tools” exists in the Eclipse Java Development Tools (JDT) project. This being the case, the DTP scope says that such tools are data-dependent and not within the scope of DTP. Essentially, the DTP scope section says something like: “DTP includes data-centric frameworks and tools not having a more specific problem domain within existing eclipse.org projects.” While this test is not perfect, relying on reasonable interpretation in the spirit of the language, we felt it offered the clearest test for what properly belongs in DTP and what does not.

DTP Projects

DTP is a top-level project. As such, it is a container for a number of subprojects adhering to the DTP scope. For convenience (though introducing some confusion), these subprojects are typically referred to as “projects” as well. I will use this convention in this article.
    
Currently DTP consists of three projects: Model Base, Connectivity, and SQL Development Tools. Based on community interest and support, the first version of DTP will lean toward relational database frameworks and tools, though recall from above that DTP is not strictly limited to this area. In fact, the Connectivity project will be generic even in the first release, allowing for extensions by the community that fall outside relational database components. In the following paragraphs, I will describe the contents of each of these projects. The intention here is not to provide component level detail or code because DTP is currently in the formative stage. Rather, I concentrate on frameworks and tools planned for the first release. Details, including current status for DTP and its projects can be found at the DTP web site (http://www.eclipse.org/datatools). Release time lines will be described in the next section.
    
The Model Base project provides domain models for use within DTP and DTP extenders. These models, built using the Eclipse Modeling Framework (EMF), support definition of drivers, databases, and SQL structure. Driver definition is used by the Driver Management Framework discussed below, the database definition delineates properties of particular database systems, and those for SQL deal both with SQL statements and queries. Thus, Model Base is essentially a container for core data structures used in DTP and follows a model-driven approach through UML based on EMF. The concentration of models into this project creates a logical space for the functionality within DTP, and the use of model-driven development leverages the best practices of domain abstract enabled by EMF.

The Connectivity project provides frameworks for connection management and tools for using connections. The Driver Management Framework (DMF) is an extensible data access driver component based on template definitions. By using DMF, contributors can register types of data access drivers, which then can be used by any other Eclipse components requiring driver locations and other relevant information. A key feature is enabling users to enter such information once, and reuse it across data tools in Eclipse. Building on DMF, the Connection Management Framework (CMF) defines specific connections through configuration data and provides connection instances. A contribution to CMF is called a ‘Connection Profile’ and can be associated with whatever data source the contributor wishes, as long as the CMF interfaces are implemented. In the initial version of DTP, exemplary Connection Profiles concentrating on JDBC will be provided to enable access to database systems.

Connection Profile instances can then be used by any components in Eclipse requiring connectivity. An example of such is the Data Source Explorer (DSE). DSE uses a tree-based UI view enabling inspection of data source contents (as provided by Connection Profile contributions). For example, a database Connection Profile might show stored procedures and tables contained within the connected database.

Finally, the Open Data Access (ODA) component provides an abstraction over the concept of ‘data,’ in the same way that CMF provides an abstraction over ‘connection.’ The ability to access data, regardless of its underlying structure, is a powerful advantage to applications seeking to use heterogeneous data sources. Indeed, ODA has been contributed to DTP from the Business Intelligence and Reporting Tools (BIRT) project, where the need for data from a variety of source is a fundamental requirement for BI and reporting applications. ODA complements the generic capabilities of CMF and the relational tools found in the SQL Development Tools project, and contributes powerful functionality to Eclipse extenders.

The SQL Development Tools project provides frameworks and tools for working with SQL. The frameworks concentrate on making database system capabilities available to tools in a way that is extensible for the specifics of particular database systems. The Routine Editor and Routine Debugger frameworks provide way of interacting with stored procedure and functions, the SQL Query Parser enables SQL tools to use a parse tree associated with a SQL query and perhaps specific SQL dialect, and the SQL Execution Plan framework enables understanding and tuning of SQL queries.

Building on the frameworks in SQL Development Tools, a number of components provide end user functionality for working with SQL. The SQL Editor is a text-based Eclipse editor having the standard colorization, syntax recognition, and code assist capabilities common through Eclipse source editors. A higher level of abstraction is provided by the Visual SQL Query Builder, which provides a graphical metaphor for SQL statements. The Results View shows the output of executing a particular SQL statement against a given database (associated with a Connection Profile), and the Script History provides view of session queries for quick re-use and tuning.

Timelines for DTP

As mentioned in the introduction, DTP is still in the planning stage, so no definitive time lines have been settled upon. There are some initial projections, however, based on a series of milestone releases. Currently the plan is to have four milestone release, two in the fourth quarter of 2005, and two more in the first quarter of 2006. As is typical with eclipse.org projects, the final milestone release would then be thoroughly tested and validated by the community before being declared the release package.
    During each milestone release, we plan to increase functionality and stabilize gains made in previous milestones. The intention is to get working components into the hands of the community as early as possible, allowing for an agile feedback loop to be established and acted upon. After the first release, DTP would then move to solidify API commitments based on community feedback, aiming for a later release addressing this area. More, and current, detail about current DTP plans can be found at the DTP web site and on the DTP mailing lists.

Conclusion

DTP offers a great opportunity to the Eclipse community to build a set of frameworks and tools that we can all leverage in our applications. Like any other open source project, the contents, time frames, and direction of the DTP project depends on contributions from the DTP community itself. These contributions are, again in the spirit of open source, based on meritocracy and can range from casual conversation to deep involvement in actually building out DTP. We at DTP welcome such community involvement, and look forward to the collaboration.