GRAPHICS DESIGNER (1988-1993)

Introduction

Albeit their extensive graphics functionality, current graphics systems lack important features. When creating graphics, much time has to be spent with the boring task of arranging, aligning and re-sizing graphics elements globally and relative to each other. There are powerful presentation packages available which deliver nice diagrams. However, the user is left alone with the decision which type of chart to choose and which graphical means to apply.

The GRAPHICS DESIGNER implements basic graphics design knowledge that enables the user to be freed from boring routine tasks and given guidance in the use of graphics presentation means. Graphics design allows for a great variety of valid solutions. Rather than enforcing a certain design policy or asking the user for a complete specification of the desired result, the graphics designer uses its design knowledge in conjunction with the given data and user-specified intentions to eliminate invalid and to offer recommended solutions.

The challenge of the project is the acquisition of a non-trivial amount of graphics design knowledge in a form suitable for machine processing and the application of methods for reasoning under uncertainty to cope with under specified or contradictory problem description.

To achieve this task, a flexible object-oriented graphics system has to be developed, which gives access to all graphics attributes under consideration, and to combine it with artificial intelligence tools which perform reasoning about features present in the graphics and derive probable intentions of the user.

The GRAPHICS DESIGNER is an enabling project which tests the viability of different approaches. It shows typical functionality to be expected from graphics systems in the future. The results can only be achieved by a successful interdisciplinary work of experts in the area of Computer Graphics and Artificial Intelligence. Therefore this activity has been positioned within the joint project TASSO (inexact acronym for "technical assisting system for processing inexact knowledge"; TASSO was partially funded by the German Federal Ministry for Research and Technology under number ITW 8900 A7).

The project was started in 1988 and ended in 1993. The work concentrated on selected areas to allow for a sufficient deep study and prove of concepts by prototype implementations. Four prototypes have been developed as components of the GRAPHICS DESIGNER: beautifier, diagram generator, graphical search, and Gestalt recognizer.

BEAUTIFIER

The BEAUTIFIER focuses on the analysis of a draft graphics for inconsistencies in positions, sizes, alignments and stylistic elements and the correction of found irregularities. Typical error situations in graphics are predefined using a flexible situation language developed as part of the project. This language can be used to add further situations if required. The BEAUTIFIER detects all errors in a graphics and builds up a network of graphical constraints describing the desired correct states. As the constraint set may be too big to be handled or even inconsistent, a new technique to handle constraints had to be developed which allows for a piecemeal solution of the constraint problem. Contradictions, which might arise between constraints are handled by a non-monotonic reasoning system EXCEPT. If the system cannot cope appropriately with some aspect of the graphics, the user is asked for help according to the assistant paradigm of the AC.

The effects of the BEAUTIFIER on a simple drawing

DIAGRAM GENERATOR

The DIAGRAM GENERATOR accepts data with attributes and derives appropriate diagrams. Attributes may include type (e.g., nominal, ordinal, cardinal), meaning (e.g., time, currency), or intentions (e.g., "emphasize maximum value"). Such characterized data can be specified by the user in a spread sheet or are generated automatically by systems like the statistics interpreter EXPLORA which actually was used for generating example data sets.

In order to derive possible diagram formats from given data, the prototype uses non-monotonic reasoning techniques to generate plausible solutions from imprecise goal descriptions. The rules extract specific characteristics in data and assign the corresponding graphical features. Finally, the assembly of features is combined into a non-contradictory presentation, that is proposed to the user and may be further refined by him. The use of non-monotonic reasoning allows the user to make incomplete and imprecise statements as is typical for human-human dialogue. In contradiction to the usual appliance of expert systems, we focused on a competent system that helps an autonomous user to cope with his tasks. The principle of autonomy still allows the user to freely express his creativity. Thereby this system is of interest for casual as well as expert users.

This is the first time that non-monotonic reasoning techniques have been applied successfully in a realistic scenario (i.e., the generation of business charts) with a non-trivial rule-set.

Chart generation: From input data to finished chart

GRAPHICAL SEARCH

Graphical search is an important counterpart to the generative components of the GRAPHICS DESIGNER. This function complements the classical keyword-oriented queries by giving the user the possibility of drawing a rough sketch of significant features of the desired graphics. A sketch is--by definition--rough and imprecise; the retrieval mechanism working on such a query has to cope with vagueness in position, size, number, etc. of the graphical objects within the sketch. Even the type of the objects may be inexact: a rectangle can be one integral object, combined from four lines, or drawn by freehand input.

A graphical retrieval based on an inexact sample is also an appropriate means of finding relevant examples for a design task. The user sketches the major components of a graphics and the search component finds similar graphics by searching a graphical data base with professionally designed samples. This facility complements the generation and beautification components in areas where no relevant knowledge could be acquired and formally represented.

Search by sketch

GESTALT RECOGNIZER

The GESTALT RECOGNIZER is able to find characteristic elements within a graphics. In psychology, such an element is called "Gestalt". An example of a Gestalt is a square-shaped arrangement of circles. The GESTALT RECOGNIZER allows for a new level of analysis of a graphics as a Gestalt reflects more closely what a user sees in and remembers from a graphics than just specific graphics primitives and attributes.

The Gestalt structures emerge as a result of principles by which the human visual system organizes visual stimuli. These principles were discovered at the beginning of this century by psychologists adhering to the Gestalt school of psychology. The Gestalt psychologists expressed these principles as a set of non-quantitative Gestalt laws. Several machine vision systems have been built which simulate some of the Gestalt laws. The quantification of the Gestalt laws turned out to be a central problem.

A prototype has been implemented which combines several models of visual perception which yield a basis for the quantification of the Gestalt principles. It recognizes many of the well-known Gestalt test patterns correctly and is able to find salient elements of presentation graphics.

These elements can be used by the BEAUTIFIER to allow for a beautification on a higher level. They can also be used as cognitive adequate search patterns for content-based graphical search. The recognition task--a prerequisite for graphical search--requires a classification of Gestalt groupings. In order to achieve this, we have investigated the use of fuzzy logic and machine learning techniques such as conceptual clustering.

Examples of groupings detected by the Gestalt recognizer.

EPICT

All prototypes rely heavily on an object-oriented graphical language (EPICT) which we have developed to provide the vocabulary needed to express all graphical features used by the analysis and reasoning components. The close coupling of a graphics system with reasoning components is a unique characteristic of this graphics system. The implementation of a graphical editor (GREAT), which provides the full set of EPICT constructs in an interactive environment, was carried out in conjunction with a software house specialized in graphics software.

All components have been implemented with a realistic set of functionality and have been introduced to a broad public on Hannover fairs CeBIT '92 and CeBIT '93.

Selected Papers

Dieter Bolz. Maschinelle Assistenz bei der Korrektur von Zeichnungen, München: Oldenbourg Verlag (1994), ISBN 3-486-23043-3. Zugl. Dissertation Universität Bremen, 1993.

Dieter Bolz. Some Aspects of the User Interface of a Knowledge Based Beautifier for Drawings. In: Wayne D. Gray, William E. Hefley, and Dianne Murray, eds Proc. 1993 Int'l Workshop on Intelligent User Interfaces, ACM Press, New York, 1993.

Dieter Bolz, Tom Gordon, Joachim Hertzberg, Josef Schneeberger. Planen unter Unsicherheit und planbasiertes Verschönern von Grafiken. In: F. di Primio, ed Methoden der Künstlichen Intelligenz für Grafikanwendungen, Bonn: Addison-Wesley (1995) 115-158. ISBN 3-89319-873-3.

Dieter Bolz. Eine Sprache zur Beschreibung graphischer Situationen, TASSO-Report 13, GMD, Sankt Augustin, 1990.

Peter Henne, Günther Schmitgen. Assoziatives Graphisches Suchen. Konzepte und Realisierung der Suchfunktion SkaGra. In: F. di Primio, ed Methoden der Künstlichen Intelligenz für Grafikanwendungen, Bonn: Addison-Wesley (1995) 183-208. ISBN 3-89319-873-3.

Peter Henne, Günther Schmitgen. Grafische Suche: Benutzerwünsche, Einsatzbereiche, Repräsentationsformen, Methoden. TASSO-Report Nr. 34. GMD, Sankt Augustin, 1991.

Peter Henne, Günther Schmitgen. Grafisches Suchsystem SkaGra: Konzepte und Realisierungen. TASSO-Report 49, GMD, St. Augustin, 1993.

Klaus Kansy, Dieter Bolz, Rüdiger Kolb, Günther Schmitgen. Graphics Designer. In: Hoschka, Peter, ed Computers as Assistants - A New Generation of Support Systems, Hillsdale, New Jersey, USA: Lawrence Erlbaum Associates, Inc. (1996) 112-126. ISBN 0-8058-2187-2.

Klaus Kansy. Leitbeispiel Graphikdesigner, TASSO-Report 14, GMD, Sankt Augustin, 1990.

Klaus Kansy. Der Grafikdesigner - wissensbasiertes Erzeugen und Verschönern von Grafiken. Office Management 39(10):25-28, 1991.

Rüdiger Kolb. Diagen - Ein Diagrammgenerator auf der Basis nichtmonotoner Logik. In: F. di Primio, ed Methoden der Künstlichen Intelligenz für Grafikanwendungen, Bonn: Addison-Wesley (1995) 61-78. ISBN 3-89319-873-3.

Rüdiger Kolb. Die graphische Umsetzung statistischer Aussagen, TASSO-Report 39, GMD, Sankt Augustin, 1992.

Rüdiger Kolb. Applying Non-Monotonic Reasoning to Graphics Design, TASSO-Report 52, GMD, Sankt Augustin, 1993.

Erich Rome. Simulierte Gestalt-Erkennung in Präsentationsgrafiken, Reihe DISKI. Sankt Augustin: infix. ISBN 3-89601-099-9 (1995). Zugl. Dissertation Universität Bremen (1994).

Erich Rome. Maschinelle Gestalt-Erkennung in Präsentationsgrafiken, Kognitionswissenschaft, vol. 5, no. 2, 82-94. (1965).

Erich Rome. Grafiksuche und Gestalt-Erkennung. In: F. di Primio, ed Methoden der Künstlichen Intelligenz für Grafikanwendungen, Bonn: Addison-Wesley (1995) 209-225. ISBN 3-89319-873-3.

Rome, Erich. Graphical Search Assistance. In: Hoschka, Peter, ed Computers as Assistants - A New Generation of Support Systems, Hillsdale, New Jersey, USA: Lawrence Erlbaum Associates, Inc. (1996) 127-141. ISBN 0-8058-2187-2.

Erich Rome, Karl Wittur, Dieter Bolz. EPICT - Eine erweiterbare Grafik-Beschreibungssprache, TASSO-Report 4, GMD, Sankt Augustin, 1990.

Research staff

Dieter Bolz, Klaus Kansy, Rüdiger Kolb, Günther Schmitgen, Erich Rome

Please contact: Klaus Kansy