All applications
DataVirt
Virtual, masked databases on top of production: an acceptance test or a demo in two clicks.
DataVirt installs on your device (Windows, Linux and Android) and opens with your Prophetix account, the same one as for the web applications.
DataVirt connects to your SQL Server, PostgreSQL, MariaDB, MySQL and Db2 databases. It reads their schemas, draws their diagrams, and spots the columns that hold personal data: names, emails, phone numbers, IBANs, card or social security numbers, dates of birth, salaries. For each one, you choose how to mask it — a plausible name, a fictitious email, a shifted date, a noisy amount — and the mask stays the same from one table to the next, so joins hold.
A virtual database is created in seconds, without copying the data: each table in it is a view on production, transparent or masked, and everything the application writes stays in the virtual database. Production is never touched. You run a deployment script on it to check changes to structure and data, you reset it, you throw it away — or it deletes itself at the set time.
A service, on Windows, on Linux or in a container, keeps the connections on your side. Its API controls everything from a pipeline (GitHub, GitLab, Azure DevOps) or a ticketing tool (Jira, ServiceNow); the interface opens in the browser, on the Windows and Linux desktop, and on Android.
DataVirt is in preparation. It will join the Prophetix subscription catalogue when it is released.
Every feature of DataVirt
65 features in 10 areas
Data sources
- Connects to your SQL Server, PostgreSQL, MariaDB, MySQL and Db2 databases.
- Declares a production database, preferably with a read-only account, and tests the connection.
- Reads the schema: tables, columns, types, keys, indexes and volumes.
- Draws the schema diagram, exportable as SVG or Mermaid.
- Detects columns holding personal data from their names and a sample of values: names, emails, phone numbers, IBANs, cards, social security numbers, IP addresses, dates of birth, salaries.
- Marks each proposal “to be confirmed” until a person validates it.
Data masking
- Chooses a masking strategy per column, among sixteen: empty or fixed value, hash, last name, first name, full name, email, phone, address, city, IBAN, card, partial, shifted date, noise, text.
- Makes masking deterministic: the same value gives the same mask, so joins, groupings and uniqueness hold.
- Compares original and masked values in a preview, to tune the rules before handing over the database.
- Keeps the type and length of each column, and leaves empty values empty.
- Gives each database its own salt, so the masks of one database do not match those of another.
Pseudonymization, anonymization and replacement
- Pseudonymizes numbers without collisions, so a primary key stays unique.
- Makes the pseudonym follow the foreign keys that reference the column, so that joins hold.
- Builds an email from the pseudonymized first and last names of the same row.
- Gives the same pseudonyms across all the databases of a domain, to follow a customer from one acceptance-test environment to another.
- Generalizes dates, numbers and texts (year, range, first characters) to help anonymize.
- Measures the re-identification risk by k-anonymity, without and then with the rules, and says what to generalize.
- Replaces with values from your lists, correspondence tables or generated formats.
- Applies a pseudonymization, anonymization or replacement profile in one click, and checks the consistency of the rules before saving them.
- Retrieves, for administrators, the pseudonym of a value or the original value of a pseudonym, with a log entry that does not contain the original value.
Virtual databases
- Creates a virtual database in a few seconds, without copying the data: each table in it is a view on production.
- Chooses the mode: transparent, masked, or masked with a preview of the originals and the masks.
- Keeps the application's writes in the virtual database; production is never written to.
- Gives the application an account specific to the database, which does not read the originals in masked mode.
- Exposes each database on a port of the agent's machine, like a real server: the application keeps its usual driver.
- Chooses the port of a database, including that of the engine, so that the agent can serve as a jump host.
- Gives a database an expiry date: it deletes itself at the set time.
- Extends, resets and deletes a database, and changes its mode.
- Materializes tables into real copies so that they can be written to or changed in structure.
- Lists the tables that are read-only, without a primary key.
Deployment scripts
- Runs a deployment script on a virtual database to check changes to structure and data.
- Splits the script according to the engine and runs it statement by statement.
- Returns a report: statements, materialized tables, structure differences from the source and writes table by table.
- Fails the pipeline, with a 422 response, when a statement fails.
Impact analysis
- Measures a set of read queries before and after a change: index added or removed, column removed, renamed or retyped, data migration.
- Gives a verdict per query: identical, faster, plan changed, slower, result changed or broken.
- Shows the rows that appeared and disappeared when a result changes.
- Lists the views, functions, procedures, triggers, keys and indexes that depend on what the script touches.
- Simulates a change, then resets the database to zero.
- Saves the set of queries on the source, for all its databases, and updates it from your repository.
- Returns the report in JUnit format for pipelines, with a failure in case of regression.
Control from the website
- Installs an agent on your side, on a Windows, Ubuntu, Debian or Linux with systemd VM, or in a container: it reaches your databases and the website over outbound HTTPS.
- Enrolls the agent with a code displayed at installation, or with a token for an automated installation.
- Controls from datavirt.prophetix.com your sources, hosts, maskings and virtual databases, with the same screens as the agent's local interface.
- Keeps your databases' passwords on the agent: the website holds none of them.
- Grants reader, operator or administrator rights on an entire agent or source by source, by email.
- Revokes an agent from the website, which cuts its channel immediately.
- Keeps a log of operations and keeps the agent's local interface as a fallback.
Integrations
- Exposes a documented REST API, which creates, resets and deletes databases from a pipeline.
- Provides examples for GitHub Actions, GitLab CI and Azure DevOps.
- Creates, resets or deletes a ticket's environment from a ticketing tool, such as Jira or ServiceNow.
- Sends signed webhooks to follow events.
- Protects access with access keys with three roles, or with sign-in to your Prophetix account.
On your devices
- Web: the datavirt.prophetix.com website and the agent's local interface, in a browser.
- Windows and Linux: a desktop application that controls a DataVirt service.
- Android: an application that controls a DataVirt service, to create an acceptance-test environment in two gestures, extend, reset, delete, view a preview and read the log.
- The service itself runs on Windows, on Linux (systemd) or in a container.
Good to know
- SQL Server and Db2: the SQL is generated and reviewed, but has never been run on real servers. PostgreSQL and MariaDB have been verified.
- On MariaDB and MySQL, written tables must be materialized: writing to a virtual table is not possible.
- Db2 requires the IBM driver on the service machine.
- The Windows service, the Android application and the real Prophetix sign-in have not been tried yet; the desktop has only been tried against a fake service, never on Linux.
- The source's foreign keys, constraints and triggers are not recreated on the views.
- Pseudonymization helps protect data, but whoever holds the key and the source can re-identify everything.
- K-anonymity does not measure the whole re-identification risk.
- The datavirt.prophetix.com website has not yet been tried in production: its deployment and the real Prophetix sign-in remain to be done.