()
()
()
()
()
()
Copyright © 2005 Nuxeo
Diffusion authorized « as is », without any modifications and including front and back covers.
Sommaire
| 1 Introduction |
| 2 CPS3 Project Motivation |
| 3 Event-driven framework |
| 4 Repository and proxies |
| 5 Document types |
| 5.1 Schemas, widgets and layouts |
| 5.2 Flexible documents |
| 6 Directories |
| 6.1 The directory abstraction |
| 6.2 User management |
| 7 Workflow-driven model |
| 8 User interface |
| 9 Development process |
| 10 Applications and adoption |
| 11 Future |
| 12 REFERENCES |
Nuxeo CPS (version 3) is an open source, GPL-licensed framework designed for building content management, collaboration and business processing applications. It addresses the needs of web applications developers to easily create rich document types, with workflow-based access control, portal-like interfaces for accessing documents and information, and rich services that match the features of best of breed non-free (proprietary) offering in the domain, including: single-sourcing, versioning, indexing, metadata management, localization and translation, notifications, commenting, theming (user interface customization), users and groups directories, content scheduling and staging, syndication... Higher level components, including mail, calendaring and discussion applications, have been developed on top of the base framework, as well as custom business applications featuring complex document types and workflows.
CPS3 has been developed by Nuxeo SARL, an innovative French company specialized in open source software and services, and a community of contributors, since 2003, and is now in use for majors projects in the Administration, in France and other countries, and in several multinational corporations. It is based on, and extends in substantial ways, the Zope application server and its Content Management Framework
Zope is an innovative application server started in 1996 and fully open sourced under the ZPL in 1998 by its developers, Zope Corporation, an American company. Zope provides an object database, an application server with an object broker that converts HTTP (or other protocols) requests to objects and methods invocation, a security model, a templating and a scripting language, a basic component model for extending the application via dedicated plugins also called “Products”.
The Content Management Framework (CMF), released by Zope Corp. in 2001, provides a component architecture and some basic services to write web content management applications: membership services, indexing, document types, and a powerful document-based workflow engine called DCWorkflow.
Nuxeo has worked on several customers projects in 2001 and 2002 using the CMF and some basic extensions, but it appeared quickly that only simple content management – mostly web publishing applications – but no truly collaborative applications with thousands of users, could be created with the CMF and basic extensions, and that some major extensions, refactorings and new model implementation were needed to compete in the so-called “ECM” (Enterprise Content Management) and collaboration applications markets dominated by Documentum, OpenText and Interwoven.
The CPS3 project was thus launched in 2003 after the company had restructured itself with a R&D department, and considerable resources were devoted to develop the new framework. First customers projects developed on top of CPS3 were in production by the end of 2003, and stable releases of the software have been issued regularly since 2004, drastically improving the framework.
In an enterprise content-management system (ECMS), many components have to be tied together to provide a high level of functionality. Some of the components are native to the framework, and some are added later as optional modules.
Many of these components have to cooperate with each other to provide their intended functionality. A simple example would be a workflow component and a mail-notification component that have to work together to send an email to some users when a document changes state in the workflow.
Whereas the existing Zope ECMS use explicit knowledge by each component of the other components that may interact with them, CPS chose an event-driven framework for these interactions. In the example above, the workflow component would send an event when the document changes state, and the mail-notification component would listen to that event, and dispatch a mail according to the policy configured for it.
This provides a great flexibility to the developer and to the integrator. The developer can choose to send an event whenever his module does a significant action, but without having direct knowledge of what other module may or may not act on this event. He can also choose to register his module as a listener for some kind of events, and decide what has to be done when an event is received. The integrator can choose what modules he installs, and knows they'll work together if it makes sense for them. Furthermore, the policy of the interaction (to whom the mail should be sent, in our example) can be clearly specified in the configuration of the listener.
Zope's object database (ZODB) lends itself naturally to hierarchical object structures, and this is how most ECMS are implemented in Zope and CMF, including earlier versions of CPS. The object orientation and the orthogonal persistence of the ZODB leads to a rapid development process, removes many of the troubles you get when using relational databases, and is more natural for document databases and document representation. But relational databases facilitate multiple orthogonal views on data in a way that hierarchical object databases don't.
In CPS, unlike the classical approach outlined by the default component Zope CMF, documents are stored in a central repository and “proxies” pointing to the documents are dispersed throughout the website structure. The user only sees proxies, and for him they look like normal documents, but the proxies are just a kind of pointer, a link to the real information. In the repository, documents are stored with an additional version number.
Because the proxies have additional metadata, a number of advanced use cases can be fulfilled:
Multi-publication: A single document can be published in different parts of the site. This is because a document can be pointed to by several unrelated proxies.
Versioning, check-in/checkout: Documents can be present in several different versions in the repository, and a proxy can point to a specific version, so different proxies can point to different versions of a single document. This makes it possible to have a “validated” version somewhere, and a “working” version somewhere else; documents don't need to be removed from publication while they are edited.
Multilingualism: A proxy can point to several documents, the one actually displayed being chosen according to the user's preferred language.
Multiple workflow states: Because the proxy objects are the ones that follow the workflow, it is possible to have the same version of a document be published in some part of the site and still be pending in another part where the local reviewer has not had time to do his work.
Multiple storages: Because the repository is the central point of access for storage, it is possible to implement and plug different storages on the repository to store data into other databases like XML or SQL databases or even plain filesystem. The repository service unifies all these backends and provides a single view of the storage space (this feature is commonly use to set up one databse per year, for example).
All these advanced features come nearly for free once you implement a proper indirection, the proxies, between the hierarchical tree and the data.
Document types are created in CPS using the abstraction of schemas and layouts. A schema describes the structure of a document as a set of typed fields. The fields may be strings, integers, lists of strings, files, or any arbitrary complex datatype. A layout specifies how a schema is to be rendered and edited, using a list of widgets that each specify the user interface and display constraints for one or more fields. New field and widget types can easily be extended by the programmers.
The “layout” abstraction is important, because then a field is not tied to a specific widget for its display. This allows changes to the View or Controller (in the MVC paradigm) completely independently of the Model. Different layouts can be used for the same schema depending on the circumstances, which makes it possible to have different “edit” and “view” pages, or have several “view” pages that display a subset of all the information available in the document – this is typically used for the Dublin Core part of the metadata for instance.
Schemas and layouts are a fundamental innovation in CPS3, in that they allow the quick development or the customization of rich document types. They are now the foundation for most of the CPS user interface, including directories (see below).
Although schemas are not a new concept in ECMS, CPS3 provided the first schema system for Zope where schemas were stored in the object database. This enabled us to create CPSTypeMaker, an innovative tool used to automatically create and modify document types on the portal. It provides a complete WYSIWYG editor that generates and edits CPS schemas and layouts definitions used by CPSDocument to create instances of a given type. This editor brings CPS's powerful document development to a higher lever, making it usable by non-developers that wish to easily extend or create document types on the portal as well as enabling rapid-prototyping. Furthermore, the definitions generated by the tool are more reliable and less buggy than human-made definitions.
In addition to obeying a schema and a layout according to their type, documents can have a “flexible” part that can be changed dynamically by the user. The user is able to manipulate a given instance of a document and add new fields and widgets to it. Using this, the user changes the schema of his document easily, for instance by adding several images or different text zones, and he can also change the way a given field is displayed, by changing the configuration of the widget associated to it, or by changing the widget completely.
Not all sites want this of course, and this behavior can be turned on and off per document type.
It has been recognized early in the development of CPS that not all data manipulated by an ECMS can be considered as a “document” from the user or infrastructure's point of view. For instance this is the case for the information about the users themselves, the way they are organized into groups. This “non-document” data has several common features:
it is not workflowed or versioned,
it is stored in an external storage (LDAP, SQL),
it has to be searched fast,
it still obeys a clear schema.
The “directory” abstraction of CPS covers these use cases. A directory contains entries that obey to a schema and are displayed using a layout, thus reusing all the framework described above for documents (indeed, when displayed and edited by a user there may be no difference at all between a document and a directory entry).
A number of backends have been written to cover the common use cases of storing the data in an LDAP directory, in an SQL table, or even of course in the ZODB itself. Additional backends provide “meta-directories”, because they can aggregate different directories and make them appear as a single one.
Using the directory abstraction, it becomes much easier to manage users because the directories expose a single unified API. The standard user management components (called “user folders” in Zope) can be replaced by a simple one that specifies which directory to use to store users, and how to identify and authenticate them.
Using meta-directories, an arbitrary complex number of user sources can be aggregated and used seamlessly by the rest of the framework. This makes it possible to adapt to complex organization's needs. For instance some of the users may come from a read-only corporate LDAP directory, with their last login date and picture stored in an associated directory in the ZODB, and some other users may be stored in several SQL tables on different servers.
Most of CPS behavior related to documents is workflow-driven. CPSWorkflow extends the basic document-based DCWorkflow by providing:
local workflow configurations (documents may follow different workflows depending on where they are located in the website's arborescent structure),
specific workflow behaviors that address the needs of multi-publishing, versioning, grouping related document in the same workflow, etc.,
“stack” workflows that can be used for dynamic workflow chain allocation.
The workflow stacks are the latest and most complex of these extensions. A stack provides dynamic behavior (in the sense that they can be different for documents following the same workflow) of who is allowed to work on a document, and along what rules. A typical use case is when, during the validation of a document, the reviewer decides that he has to postpone his decision until the document has been approved by specific users. The reviewer then adds the users to a stack; these users can validate or not the document and when the stack is empty the document returns to the reviewer for his final decision.
CPS was first developed with a portal user interface based on “boxes”: graphical elements of information that either the webmaster, the users locally responsible for content management, or registered end-users, can use to customize the content and presentation of information displayed for themselves or for other users of the portal.
CPSSkins was developed at Chalmers University of Technology, then integrated in CPS, to address the shortcomings of the “boxes” interface. CPSSkins features:
a WYSIWYG interface for editing the global look of a site, also called a “theme”; an arbitrary number of themes can be created and used for a site,
a visual editor with drag and drop facility for moving information boxes, or “templets”, around the page, and reconfiguring them,
a user interface for managing “portlets”. This gives users that are not site managers the possibility to add visual content to pages while preserving the overall site design.
Using CPSSkins, a site can have its look drastically altered in a matter of minutes, either in totality or in selected sub-sites. This caters for the need of complex enterprise websites where different organizational units need different user experiences. This also makes it possible to build and distribute CPS Business Templates that allow dramatical changes to the user interface to fit business needs (for instance a public website's backoffice should look different than an online community).
CPS is developed using a “test-driven” approach. When new coded is added, unit tests or functional tests are added at the same time to ensure quality and non-regression. When a bug is isolated, a test is written for it so that in the future no regression can happen. Periodical automated processes run the whole test set and report any problem.
Nuxeo joined in 2004 the EDOS European project to further enhance its tools used to manage the quality and address the complexity of the project.
CPS is originally a project of Nuxeo but it is open source and has been extended by third-party developers. The very innovative CPSSkins project, for instance, was started independently before being merged during the CPS 3.3 development cycle. Development “sprints” (multi-day sessions of focused and intense development, based on extreme-programming ideas) have been organized to foster collaboration between Nuxeo and third-party developers.
Moreover, to ease collaboration with other parties, a community website and a bugtracking system are online and publicly available.
CPS has been used at Nuxeo, by major IT companies in France and Europe, and directly by developers that download the application, for:
major internet websites for French ministries (Ministry of Interior and Ministry of Defense), local administrations (city of Lyon, region of Brittany) and public-sector organizations (French Institute of Statutory Auditors),
major intranets for French and foreign ministries (Senegal government, French ministries of Culture and Interior),
collaborative applications for major companies (Groupe Suez, STMicroelectronics, Central Bank of West-African States),
paper and electronic mail and files management application for the ministries of Interior and Justice
civil-state management for the city of Antananarivo in Madagascar,
all the web sites of the Chalmers University of Technology in Göteborg, Sweden.
In most of these projects, majors criterions for CPS selection by customers are first that CPS offers a framework competing with market leaders' proprietary solutions and, secondly, the open source licence benefits. Most of those customers value the higher adaptability of the software, the long term strategic independence it guarantees, the benefits brought by mutualization between similar projects carried out for different customers, and the lower global price when the application has to scale to tens of instances and tens of thousands of users.
Zope is currently undergoing a transition from the now classical Zope 2 to the very innovative, fully component-based Zope 3. Although architecturally superior to Zope 2, Zope 3 currently lacks the content management facilities provided by the CMF.
Work has been started in autumn 2004 at Nuxeo and other companies to gradually port or migrate the functionalities of CPS3 to the Zope3 framework. It is expected that a fully Zope 3 based version of CPS will see the light before the end of 2005.
French Institute of Statutory Auditors
Institute for Public Management and Economic Development (French Ministry of the Economy, Finance and Industry)
French Ministry of Defense portal
Federation of the Belgian Chemical Industries website
Portal of Culture in France (Ministry of Culture)