SDK Status

All enrich application development now uses the Enrich SDK. The SDK allows unit testing of transforms. The features and documentation is growing but should be considered experimental. It is being used/tested by close partners of Scribble Data.

Supported Features

+-----------------+--------------------+---------------------------------------+
| Feature         | Status             | Notes                                 |
+=================+====================+=======================================+
| Transform       | Stable             | Bootstrap, Test                       |
+-----------------+--------------------+---------------------------------------+
| Tasks           | Stable             | Bootstrap, Test                       |
+-----------------+--------------------+---------------------------------------+
| Pipeline        | Initial            | Basic testing for both tasks s        |
|                 |                    |                                       |
|                 | > and pipeline     |                                       |
+-----------------+--------------------+---------------------------------------+
| Asset           | Stable             | Bootstrap                             |
+-----------------+--------------------+---------------------------------------+
| Skin Services   | Dropped Deprecated | Use Apps                              |
+-----------------+--------------------+---------------------------------------+
| Apps            | In Development     | Django apps                           |
+-----------------+--------------------+---------------------------------------+
| Workflows       | Beta               | Prefect                               |
+-----------------+--------------------+---------------------------------------+
| Datasets        | In Development     | Dataset abstraction Will be supported |
|                 |                    |                                       |
| Notebook        | Deprecated         |                                       |
+-----------------+--------------------+---------------------------------------+
| Policy          | Deprecated         | Will be dropped                       |
+-----------------+--------------------+---------------------------------------+
| API             | Stable             | Access server                         |
+-----------------+--------------------+---------------------------------------+
| Metadata        | Stable             | Generation, export                    |
+-----------------+--------------------+---------------------------------------+
| Metadata Server | Alpha              | Generic interface                     |
+-----------------+--------------------+---------------------------------------+

Roadmap

We would like to expand the capability of the SDK. Some of the capabilities in the pipeline include:

1. Privacy support: Add more support privacy enhacing features including attribute markup and anonymization.
2. Realtime support: Expand ability to implement realtime pipelines in a fast, standardized, and consistent way.
3. Metadata support: Standardized metadata for discovery within and across organizations (as required by some privacy laws)
4. Data Quality: Support for custom expectations and simpler specifications.
5. Built-in Apps: Document the packages from enrich-scribble that are deployed by default along with the SDK
6. Notebook Support: Support data consumption and model development in the notebook context.
7. App Testing: Enable local app development by simulating the server.