KDE Commit-Digest
History
Predicate is a database abstraction layer that originated from the Kexi project. It is scheduled to eventually succeed KexiDB 2 within the Kexi project. Other projects are welcome to make use of it as well.
Back in 2002, only MySQL and CQL++ were supported by Kexi. Very few database access features were abstracted and much of the databases-specific code was within the Kexi application code itself. In 2003, the CQL++ backend was replaced by SQLite. The need for major changes was taken as an opportunity to develop an abstraction layer, called KexiDB. This abstraction allowed the developers to keep the code base of the application clean and more easily maintainable. KexiDB was a success and in 2004 SQLite (2.8 at the time) was chosen as Kexi's default database engine, to enable the user to start working without any knowledge of databases, and, more importantly, to diminish the administrative overhead. Over time, more and more databases were supported by KexiDB.
The introduction of Qt4 and the development of the KDE4 dev platform necessitated a port of Kexi to Qt4 and the KDE4 libraries. KexiDB was first cleaned-up and later ported, resulting in version 2 of KexiDB in 2006. This version was introduced to the Kexi users with the release of Kexi 2.2 in May 2010 and continues to be used in the recently released Kexi 2.3.
Since KexiDB 2 depends on only a few KDE libraries, a Qt-only branch of KexiDB 2 was begun at Akademy 2008. This branch became known as "Predicate," as proposed by Aaron Seigo.
Principles and Features
The raison-d'être for KexiDB (and thus Predicate) is the fact that both database backends used by Kexi are very different. KexiDB offers a high level of abstraction with a focus on the different data types, behavioral differences, and the differences in feature sets of each database. More specifically, SQLite uses a liberal concept of type affinity, there are file or server based backends, and different transaction models are used.
KexiDB supports backends through plugins that are based on the KDE plugin API. There is no use of QtSQL nor QtSQL compatibility. The main difference between KexiDB and QtSQL remains the ability to create databases from scratch without executing backend-specific DDL (CREATE) statements. This is unique to KexiDB. Also, modification of the schema is beyond the scope of QtSQL. While we don't intend to criticize QtSQL for this, Kexi needed these features since it is not a database administration tool. It's a complete tool for developing database applications from scratch. Some older discussions on these topics can be found here
Parallel to the development of KexiDB, a database-related graphical layer was developed for Kexi, mainly in order to achieve tabular views of larger data sets. It supported Model/View/Delegate patterns long before Qt 4 became stable and was in wider use. Other features of Kexi taking advantage of KexiDB abstraction are the Kexi Forms, Table Designer, and Query Designer. These are modules also known by the old hardcore MS Access developers.
Since Query Designer in Kexi also supports the SQL View, KexiDB delivers its own SQL parser implementation. It accepts a small subset of SQL and can be used in non-graphical programs to introspect the schema of a given SQL query without using any backend or even executing the statement (the same parser is also a part of Predicate). Classes defining the parse tree are also part of the public API. Many extensions for the parser API are planned, e.g. supporting DDL (CREATE) queries. KexiDB is able to translate parsed statements back into SQL statements, which is done differently depending on the given database backend. Its database backend may also support a flat file format. Predicate has the same parser features as KexiDB 2.
In the meantime, Kexi Web Forms plugin have been developed as a Google Summer of Code 2008 project. Like any other, also this plugin accesses the database backend through the KexiDB abstraction. This development validated usefulness of KexiDB in client/server applications and even in 3-tier architectures. Web Forms was offered in Kexi 1.1 with its own lighttpd-based web server. That said, while Web Forms is very interesting project, it is temporarily disabled in the Kexi 2 series. This module awaits being ported to Qt4 and is in need of a new maintainer.
Kexi 2 also features a well-received Reports module, a fork of OpenRPT integrated with the KexiDB 2 database abstraction used for data access purposes.
Current State & Plans
Feature plans of KexiDB have always been following the plans defined for Kexi. This is more or less the recipe for Predicate too, until more non-Kexi users appear.
Currently, Predicate is in active development and its API is not frozen. The status of the supported drivers is the same as for Kexi 2's KexiDB. It can be checked here. The SQLite 3.x, MySQL, PostgreSQL backends are stable. One extra point for Predicate is that PostgreSQL support has been reimplemented using the libpq C library, which made the lower-quality libpqxx-based one (C++) obsolete.
KexiDB 1 supported MySQL, PostgreSQL, ODBC and SQLite 2 and SQLite 3. With the port to Qt4/KDE4, the new KexiDB 2 initially supported an increased number of backends due to a high number of active contributors to the project. Thus, KexiDB 2 originally also supported xBase (dBase), Sybase (and MS SQL Server) and Oracle backends. See the sources in SVN. During the further development KexiDB 2 and the porting to Qt 4, however, support for anything but MySQL, PostgreSQL, and SQLite 3 was dropped. This reduction in database support is still the case for the current KexiDB and also for its successor Predicate. The Predicate team is looking for maintainers for these backends. The old (early KexiDb 2) source code can be reused and reintegrated into Predicate.
In the future, the Predicate project may adopt GUI bindings to the databases that are now a part of Kexi's internal framework. This includes connection configuration facilities (settings, GUIs), tabular views for queries, and database forms, just to name a few. All this would be part of an additional library such as PredicateGui, so the core of Predicate would stay non-GUI as it is now. Decisions regarding these ideas depend on the feedback from 3rd-party developers.
Further plans for Predicate include adopting a KexiDB-based migration API (i.e. facilities for importing database schemata and/or data from pre-existing data sources into formats/backends understood by Predicate). These data sources include not only database backends but also text files like CSV, and binary formats like MS Access .mdb files. Excel spreadsheets and data embedded into ODF-based formats is also within the scope of a future migration API.
The scripting APIs for Kexi's database handling will be ported to be based on the Predicate API. This may result in bindings for Javascript or other languages as a part of the Predicate project itself.
Kexi intends to switch from KexiDB 2 to Predicate in the mid-term, most probably in version 2.5.
KexiDB 2 vs Predicate
Speaking of the differences between Predicate and the latest KexiDB, there is one design decision not yet implemented. KexiDB (and thus Kexi) stores extra database schema information (e.g. table names, column names, and types) in its own "system" tables (but in the same database). This often duplicates the native database schema information. With old versions of SQLite, it was difficult to access the native schema, but by now there's no point in maintaining this duplication. However, the idea is still valid, for example, tables can have translatable names, and queries or forms need the data (e.g. XML definition) and metadata to be stored somewhere anyway.
Summing up, Predicate allows the developer to use it in a light (just database access) or a full mode (database creation, bindings to the custom GUIs). Kexi 2 currently requires a “one-time import” even of MySQL and PostgreSQL databases before opening them. As soon as Kexi is ported to Predicate, Kexi will be able to handle databases created by other programs without the import.
Compared to KexiDB, the Predicate API is increasingly based on implicitly shared data classes (like in QString or Qt DOM), and is thus much more convenient for daily use.
"Target market"
Predicate may be interesting for the rapid development of high-quality fat-client applications based on Qt only or KDE technology. Non-Qt C++ developers may find Predicate relatively easy to adopt since the Qt-specific knowledge necessary is largely limited to Qt Core.
Since Predicate is currently mainly a building block of Kexi, a graphical application, an increasingly number of new features will be devoted to the use in data-aware GUI applications. Therefore, many tasks such as defining database schemata and queries or creating forms can be performed using Kexi without touching C++ or even without computer programming, and the remaining tasks can be performed by a skilled programmer.
Predicate can be used in middle-ware layers of software systems as well; in fact the maintainer has already employed Predicate in a commercial project.
Predicate has facilities especially useful in systems where more than one type of backend is used, i.e. data warehouses or automated data migration/integration.
After Kexi moves to Predicate, it will also become beneficial to use Predicate for the development of Kexi extensions, be it for spacial databases handling, custom data-aware views or specialized import/export routines.
There are at least two companies that supported the development of KexiDB: AutomatiX GmbH, which continues hosting kexi-project.org, supported improvements in 2003 (See this commit.) And OpenOffice Software LLC owns a lot of credits as it was the sponsor of the core developer of Kexi and KexiDB, Jarosław Staniek, from 2003 to 2007.
