Try the NEW Quickstart Guide

Looking for a faster way to get started? Check out our new Quickstart Guide with an automated installer and streamlined setup process!

Getting Started

This guide will walk you through setting up a new platform locally and walk you through some of the basic concepts within the OpenTDF platform.

Pre-requisites

A copy of the otdfctl CLI.
- Download the latest release
A tool to run the compose file.
- Podman
- Docker Compose

Update /etc/hosts

In order for the services to communicate correctly you will need to update your /etc/hosts file.

echo -e "127.0.0.1 platform.opentdf.local\n127.0.0.1 keycloak.opentdf.local" | sudo tee -a /etc/hosts

Getting the Platform Running

The first step is to get the platform running locally. You can use the following docker compose file to get the platform running. This docker compose file will start a local instance of the platform, Keycloak, and a Postgres database.

warning

Not for production use.

Download docker-compose.yaml

Docker Compose

networks:
  default:
    name: opentdf_platform

configs:
  caddy_config:
    content: |
      {
        log {
            level INFO
            output stdout
        }
      }
      https://keycloak.opentdf.local:9443 {
        tls internal
        reverse_proxy keycloak:8888
      }
      https://platform.opentdf.local:8443 {
        tls internal
        reverse_proxy {
            to h2c://platform:8080
            transport http {
                versions h2c 2 1.1  # Enable gRPC proxying
            }
        }
      }

services:
  caddy:
    image: caddy:2.8.4-alpine
    command: ['caddy','run', '--config', '/etc/caddy/Caddyfile']
    configs:
      - source: caddy_config
        target: /etc/caddy/Caddyfile
    ports:
      - "8443:8443"
      - "9443:9443"
      - "2019:2019"
    volumes:
      - caddy_data:/data
    depends_on:
      ensure-permissions:
        condition: service_completed_successfully
    healthcheck:
      test: ["CMD", "wget", "--spider", "-q", "http://127.0.0.1:2019/metrics"]
      interval: 5s
      timeout: 5s
      retries: 3
    restart: unless-stopped

  check-certs:
    image: alpine:3.23
    volumes:
      - type: volume
        source: caddy_data
        target: /etc/ssl/certs
        volume:
          subpath: caddy/certificates/local/keycloak.opentdf.local/
    command:
      - sh
      - -c
      - |
        echo "Checking certificates"
        ls -alh /etc/ssl/certs
        cat /etc/ssl/certs/keycloak.opentdf.local.crt
    depends_on:
      caddy:
        condition: service_healthy
      ensure-permissions:
        condition: service_completed_successfully
    restart: "no"

  ensure-permissions:
    image: alpine:3.23
    command:
    - 'sh'
    - '-c'
    - |
      chmod -R 755 /data
    volumes:
      - caddy_data:/data
    restart: "no"

  keycloak:
    volumes:
    - keys:/keys:ro
    image: keycloak/keycloak:25.0
    restart: always
    depends_on:
      fix-keys-permissions:
        condition: service_completed_successfully
    command:
    - "start-dev"
    - "--verbose"
    - "-Djavax.net.ssl.trustStorePassword=password"
    - "-Djavax.net.ssl.HostnameVerifier=AllowAll"
    - "-Djavax.net.ssl.trustStore=/keys/ca.jks"
    - "--spi-truststore-file-hostname-verification-policy=ANY"
    environment:
      KC_PROXY: edge
      KC_HTTP_RELATIVE_PATH: /auth
      KC_HOSTNAME_STRICT: "false"
      KC_HOSTNAME_STRICT_BACKCHANNEL: "false"
      KC_HOSTNAME_STRICT_HTTPS: "false"
      KC_HTTP_ENABLED: "true"
      KC_HTTP_PORT: "8888"
      KC_HTTPS_PORT: "8443"
      KC_HTTP_MANAGEMENT_PORT: "9001"
      KEYCLOAK_ADMIN: admin
      KEYCLOAK_ADMIN_PASSWORD: changeme
      #KC_HOSTNAME_URL: http://localhost:8888/auth
      KC_FEATURES: "preview,token-exchange"
      KC_HEALTH_ENABLED: "true"
      KC_HTTPS_KEY_STORE_PASSWORD: "password"
      KC_HTTPS_KEY_STORE_FILE: "/keys/ca.jks"
      KC_HTTPS_CERTIFICATE_FILE: "/keys/localhost.crt"
      KC_HTTPS_CERTIFICATE_KEY_FILE: "/keys/localhost.key"
      KC_HTTPS_CLIENT_AUTH: "request"
      ###
      # The following environment variable resolves SIGILL with Code 134 when running Java processes on Apple M4 chips
      # 
      # On Apple Silicon (M4 chip):
      #   export JAVA_OPTS_APPEND="-XX:UseSVE=0"
      #   docker-compose up
      # 
      # On other architectures:
      #   export JAVA_OPTS_APPEND=""
      #   docker-compose up
      # 
      # Or set directly: JAVA_OPTS_APPEND="-XX:UseSVE=0" docker-compose up
      JAVA_OPTS_APPEND: "${JAVA_OPTS_APPEND:-}"
      ###
    # ports:
    #   - "${KC_EXPOSE_PORT:-8443}:8443"
    #   - "${KC_EXPOSE_PORT_HTTP:-8888}:8888"
    #   - "${KC_EXPOSE_PORT_MGMT:-9001}:9001"
    healthcheck:
      test:
        - CMD-SHELL
        - |
          [ -f /tmp/HealthCheck.java ] || echo "public class HealthCheck { 
            public static void main(String[] args) throws java.lang.Throwable { 
              javax.net.ssl.HttpsURLConnection.setDefaultHostnameVerifier((hostname, session) -> true); 
              javax.net.ssl.SSLContext sc = javax.net.ssl.SSLContext.getInstance(\"SSL\"); 
              sc.init(null, new javax.net.ssl.TrustManager[]{ 
                new javax.net.ssl.X509TrustManager() { 
                  public java.security.cert.X509Certificate[] getAcceptedIssuers() { return null; } 
                  public void checkClientTrusted(java.security.cert.X509Certificate[] certs, String authType) {} 
                  public void checkServerTrusted(java.security.cert.X509Certificate[] certs, String authType) {} 
                } 
              }, new java.security.SecureRandom()); 
              javax.net.ssl.HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory()); 
              java.net.HttpURLConnection conn = (java.net.HttpURLConnection)new java.net.URL(args[0]).openConnection(); 
              System.exit(java.net.HttpURLConnection.HTTP_OK == conn.getResponseCode() ? 0 : 1); 
            } 
          }" > /tmp/HealthCheck.java && java ${JAVA_OPTS_APPEND} /tmp/HealthCheck.java http://localhost:8888/auth 2>/dev/null
      interval: 10s
      timeout: 10s
      retries: 10
      start_period: 3m
  opentdfdb:
    image: postgres:15-alpine
    restart: always
    user: postgres
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: changeme
      POSTGRES_DB: opentdf
    healthcheck:
      test: ["CMD-SHELL", "pg_isready"]
      interval: 5s
      timeout: 5s
      retries: 10

  jaeger:
    image: jaegertracing/all-in-one:latest
    environment:
      COLLECTOR_OTLP_ENABLED: "true"
    ports:
      - "16686:16686"   # Web UI
      - "4317:4317"     # OTLP gRPC
      - "4318:4318"     # OTLP HTTP
      - "14250:14250"   # Model/collector gRPC
    profiles:
      - tracing
    restart: always

  # Provision Keycloak with initial configuration
  platform-provision-keycloak:
    image: registry.opentdf.io/platform:nightly
    command: ["provision", "keycloak", "-e", "https://keycloak.opentdf.local:9443/auth", "-f", "/configs/keycloak_data.yaml"]
    depends_on:
      keycloak:
        condition: service_healthy
      opentdfdb:
        condition: service_healthy
      patch-platform-config:
        condition: service_completed_successfully
      download-keycloak-data:
        condition: service_completed_successfully
      generate-keys:
        condition: service_completed_successfully
    volumes:
      - configs:/configs:ro
      - keys:/keys:ro
    environment:
      - OPENTDF_CONFIG_FILE=/configs/opentdf.yaml
    restart: "no"
    extra_hosts:
      - "keycloak.opentdf.local:host-gateway"
  # Prepare fixtures directory structure - create symlink to expected location
  prepare-fixtures:
    image: alpine:3.23
    volumes:
      - configs:/configs
    depends_on:
      download-fixtures:
        condition: service_completed_successfully
    command:
      - sh
      - -c
      - |
        mkdir -p /configs/service/internal/fixtures
        cd /configs
        ln -sf /configs/service/internal/fixtures ./service
    restart: "no"

  # Add sample attributes and metadata  
  platform-provision-fixtures:
    image: registry.opentdf.io/platform:nightly
    command: ["provision", "fixtures", "--config-file", "/configs/opentdf.yaml"]
    working_dir: /configs
    depends_on:
      platform-provision-keycloak:
        condition: service_completed_successfully
      opentdfdb:
        condition: service_healthy
      prepare-fixtures:
        condition: service_completed_successfully
      generate-keys:
        condition: service_completed_successfully
    volumes:
      - configs:/configs:ro
      - keys:/keys:ro
    restart: "no"

  # Prepare CA certificates bundle with Caddy cert
  prepare-ca-certs:
    image: alpine:3.23
    volumes:
      - type: volume
        source: caddy_data
        target: /caddy-certs
        read_only: true
        volume:
          subpath: caddy/certificates/local/keycloak.opentdf.local
      - platform_certs:/etc/ssl/certs
    depends_on:
      caddy:
        condition: service_healthy
    command:
      - sh
      - -c
      - |
        # Install ca-certificates package
        apk add --no-cache ca-certificates
        # Copy Caddy certificate to CA bundle
        cp /caddy-certs/keycloak.opentdf.local.crt /usr/local/share/ca-certificates/
        update-ca-certificates
        # Copy the updated CA bundle to shared volume
        cp -r /etc/ssl/certs/* /etc/ssl/certs/
        echo "CA certificates prepared successfully"
    restart: "no"

  # Main OpenTDF Platform server
  platform:
    image: registry.opentdf.io/platform:nightly
    command: ["start", "--config-file", "/configs/opentdf.yaml"]
    depends_on:
      platform-provision-fixtures:
        condition: service_completed_successfully
      keycloak:
        condition: service_healthy
      opentdfdb:
        condition: service_healthy
      generate-keys:
        condition: service_completed_successfully
      prepare-ca-certs:
        condition: service_completed_successfully
    ports:
      - "8080:8080"
    volumes:
      - configs:/configs:ro
      - keys:/keys:ro
      - platform_certs:/etc/ssl/certs:ro
    extra_hosts:
      - "keycloak.opentdf.local:host-gateway"
    restart: unless-stopped

  # Initialize volume permissions
  init-volumes:
    image: alpine:3.23
    volumes:
      - configs:/configs
      - keys:/keys
    command:
      - sh
      - -c
      - |
        chmod 777 /configs /keys
        mkdir -p /configs/service/internal/fixtures
        chmod -R 777 /configs
    restart: "no"

  # Fix keys permissions after generation
  fix-keys-permissions:
    image: alpine:3.23
    volumes:
      - keys:/keys
    depends_on:
      generate-keys:
        condition: service_completed_successfully
    command:
      - sh
      - -c
      - |
        chmod -R 755 /keys
        chmod 644 /keys/*
    restart: "no"

  # Download platform configuration file
  download-platform-config:
    image: alpine:3.23
    volumes:
      - configs:/configs
    depends_on:
      init-volumes:
        condition: service_completed_successfully
    command: ['wget', '-O', '/configs/opentdf.yaml', 'https://raw.githubusercontent.com/opentdf/platform/main/opentdf-example.yaml']
    restart: "no"

  # Patch platform configuration to use keycloak.opentdf.local:9443
  patch-platform-config:
    image: alpine:3.23
    volumes:
      - configs:/configs
    depends_on:
      download-platform-config:
        condition: service_completed_successfully
    command:
      - sh
      - -c
      - |
        apk add --no-cache sed
        sed -i 's|http://keycloak:8888|https://keycloak.opentdf.local:9443|g' /configs/opentdf.yaml
        echo "Patched opentdf.yaml to use keycloak.opentdf.local:9443"
    restart: "no"

  # Download Keycloak provisioning data
  download-keycloak-data:
    image: alpine:3.23
    volumes:
      - configs:/configs
    depends_on:
      init-volumes:
        condition: service_completed_successfully
    entrypoint: /bin/sh
    command:
      - -c
      - |
        URL='https://raw.githubusercontent.com/opentdf/platform/main/service/cmd/keycloak_data.yaml'
        OUTPUT='/configs/keycloak_data.yaml'
        MAX_ATTEMPTS=3
        
        for i in $$(seq 1 $$MAX_ATTEMPTS); do
          echo "Attempt $$i of $$MAX_ATTEMPTS: Downloading keycloak_data.yaml..."
          
          if wget -O "$$OUTPUT" "$$URL"; then
            echo "Download successful"
            
            # Validate the downloaded file
            if [ -f "$$OUTPUT" ] && [ -s "$$OUTPUT" ]; then
              if head -1 "$$OUTPUT" | grep -q -E '^(---|\w+:)'; then
                echo "Validation passed: File exists, non-empty, and appears to be valid YAML"
                exit 0
              else
                echo "Validation failed: File does not appear to be valid YAML"
                rm -f "$$OUTPUT"
              fi
            else
              echo "Validation failed: File is missing or empty"
            fi
          else
            echo "Download failed (attempt $$i)"
          fi
          
          if [ $$i -lt $$MAX_ATTEMPTS ]; then
            echo "Retrying in 2 seconds..."
            sleep 2
          fi
        done
        
        echo "ERROR: Failed to download and validate keycloak_data.yaml after $$MAX_ATTEMPTS attempts"
        exit 1
    restart: "no"

  # Download fixtures data
  download-fixtures:
    image: alpine:3.23
    volumes:
      - configs:/configs
    depends_on:
      init-volumes:
        condition: service_completed_successfully
    command: ['wget', '-O', '/configs/service/internal/fixtures/policy_fixtures.yaml', 'https://raw.githubusercontent.com/opentdf/platform/main/service/internal/fixtures/policy_fixtures.yaml']
    restart: "no"

  # Download init-temp-keys script
  download-init-script:
    image: alpine:3.23
    volumes:
      - configs:/configs
    depends_on:
      init-volumes:
        condition: service_completed_successfully
    entrypoint: /bin/sh
    command:
      - -c
      - |
        URL='https://raw.githubusercontent.com/opentdf/platform/main/.github/scripts/init-temp-keys.sh'
        OUTPUT='/configs/init-temp-keys.sh'
        MAX_ATTEMPTS=3
        
        for i in $$(seq 1 $$MAX_ATTEMPTS); do
          echo "Attempt $$i of $$MAX_ATTEMPTS: Downloading init-temp-keys.sh..."
          
          if wget -O "$$OUTPUT" "$$URL"; then
            echo "Download successful"
            
            # Validate the downloaded file
            if [ -f "$$OUTPUT" ] && [ -s "$$OUTPUT" ]; then
              if head -1 "$$OUTPUT" | grep -q '^#!/'; then
                echo "Validation passed: File exists, non-empty, and appears to be a shell script"
                exit 0
              else
                echo "Validation failed: File does not appear to be a valid shell script"
                rm -f "$$OUTPUT"
              fi
            else
              echo "Validation failed: File is missing or empty"
            fi
          else
            echo "Download failed (attempt $$i)"
          fi
          
          if [ $$i -lt $$MAX_ATTEMPTS ]; then
            echo "Retrying in 2 seconds..."
            sleep 2
          fi
        done
        
        echo "ERROR: Failed to download and validate init-temp-keys.sh after $$MAX_ATTEMPTS attempts"
        exit 1
    restart: "no"

  # Generate keys without Docker dependency
  generate-keys:
    image: alpine:3.23
    volumes:
      - configs:/configs
      - keys:/keys
    depends_on:
      download-init-script:
        condition: service_completed_successfully
      init-volumes:
        condition: service_completed_successfully
    entrypoint: /bin/sh
    command:
      - -c
      - |
        apk add --no-cache openssl openjdk11-jre bash
        cd /keys
        
        # Generate KAS RSA private key
        openssl genpkey -algorithm RSA -out /keys/kas-private.pem -pkeyopt rsa_keygen_bits:2048
        openssl rsa -in /keys/kas-private.pem -pubout -out /keys/kas-cert.pem
        
        # Generate ECC Key
        openssl ecparam -name prime256v1 > /tmp/ecparams.tmp
        openssl req -x509 -nodes -newkey ec:/tmp/ecparams.tmp -subj "/CN=kas" -keyout /keys/kas-ec-private.pem -out /keys/kas-ec-cert.pem -days 365
        
        # Generate CA
        openssl req -x509 -nodes -newkey RSA:2048 -subj "/CN=ca" -keyout /keys/keycloak-ca-private.pem -out /keys/keycloak-ca.pem -days 365
        
        # Generate localhost certificate
        printf "subjectAltName=DNS:localhost,IP:127.0.0.1" > /tmp/sanX509.conf
        printf "[req]\ndistinguished_name=req_distinguished_name\n[req_distinguished_name]\n[alt_names]\nDNS.1=localhost\nIP.1=127.0.0.1" > /tmp/req.conf
        openssl req -new -nodes -newkey rsa:2048 -keyout /keys/localhost.key -out /tmp/localhost.req -batch -subj "/CN=localhost" -config /tmp/req.conf
        openssl x509 -req -in /tmp/localhost.req -CA /keys/keycloak-ca.pem -CAkey /keys/keycloak-ca-private.pem -CAcreateserial -out /keys/localhost.crt -days 3650 -sha256 -extfile /tmp/sanX509.conf
        
        # Generate sample user certificate
        openssl req -new -nodes -newkey rsa:2048 -keyout /keys/sampleuser.key -out /tmp/sampleuser.req -batch -subj "/CN=sampleuser"
        openssl x509 -req -in /tmp/sampleuser.req -CA /keys/keycloak-ca.pem -CAkey /keys/keycloak-ca-private.pem -CAcreateserial -out /keys/sampleuser.crt -days 3650
        
        # Convert to PKCS12
        openssl pkcs12 -export -in /keys/keycloak-ca.pem -inkey /keys/keycloak-ca-private.pem -out /keys/ca.p12 -nodes -passout pass:password
        
        # Convert PKCS12 to JKS using keytool (no Docker needed)
        keytool -importkeystore \
          -srckeystore /keys/ca.p12 \
          -srcstoretype PKCS12 \
          -destkeystore /keys/ca.jks \
          -deststoretype JKS \
          -srcstorepass "password" \
          -deststorepass "password" \
          -noprompt
        
        echo "Keys generated successfully"
    environment:
      JAVA_OPTS_APPEND: "${JAVA_OPTS_APPEND:-}"
    restart: "no"

volumes:
  keys:
    name: opentdf_keys
  configs:
    name: opentdf_configs
  caddy_data:
  platform_certs:
    name: opentdf_platform_certs

Save the Docker Compose File

Click the download button above to save the docker-compose.yaml file to your local machine. If you prefer to copy the content manually, save it as docker-compose.yaml.

Start the Platform

Once you have the docker-compose.yaml file saved, start the platform using the following command:

# If running on Apple M4 chip
JAVA_OPTS_APPEND="-XX:UseSVE=0" docker compose --file docker-compose.yaml up -d

# Or on other architectures
docker compose --file docker-compose.yaml up -d

Note for Apple M4 chip users:
The JAVA_OPTS_APPEND="-XX:UseSVE=0" environment variable resolves SIGILL with Code 134 errors when running Java processes (such as Keycloak).

Trust Self Signed Certificates

After starting the platform, caddy will generate self-signed certificates during the bootstrapping process. You will either need to trust these certificates on your system or use the --tls-no-verify flag on every command. If using the --tls-no-verify command it will disable profiles and require that you pass in the host and authentication into each command.

First, extract the certificates from the container:

mkdir -p ./opentdf-certs

docker compose cp caddy:/data/caddy/certificates/local/keycloak.opentdf.local/keycloak.opentdf.local.crt ./opentdf-certs

docker compose cp caddy:/data/caddy/certificates/local/platform.opentdf.local/platform.opentdf.local.crt ./opentdf-certs

Import and Trust Certificates by Operating System

macOS

Import certificates to Keychain:

# Import both certificates to the System keychain
sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain ./opentdf-certs/keycloak.opentdf.local.crt
sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain ./opentdf-certs/platform.opentdf.local.crt

Alternative: Using Keychain Access GUI:
- Open Keychain Access (Applications → Utilities → Keychain Access)
- Drag and drop both .crt files into the System keychain
- Double-click each certificate and set Trust → When using this certificate to Always Trust
- Enter your admin password when prompted

Verify certificates are trusted:

security verify-cert -c ./opentdf-certs/keycloak.opentdf.local.crt
security verify-cert -c ./opentdf-certs/platform.opentdf.local.crt

References:

Windows

Import certificates using PowerShell (Run as Administrator):

# Import to Trusted Root Certification Authorities
Import-Certificate -FilePath ".\opentdf-certs\keycloak.opentdf.local.crt" -CertStoreLocation Cert:\LocalMachine\Root
Import-Certificate -FilePath ".\opentdf-certs\platform.opentdf.local.crt" -CertStoreLocation Cert:\LocalMachine\Root

Alternative: Using Certificate Manager GUI:
- Press Win + R, type certmgr.msc, and press Enter
- Navigate to Trusted Root Certification Authorities → Certificates
- Right-click and select All Tasks → Import
- Follow the wizard to import both .crt files
- Choose Place all certificates in the following store → Trusted Root Certification Authorities
Alternative: Using MMC (Microsoft Management Console):
- Press Win + R, type mmc, and press Enter
- Go to File → Add/Remove Snap-in
- Add Certificates snap-in for Local Computer
- Navigate to Certificates (Local Computer) → Trusted Root Certification Authorities → Certificates
- Right-click and import both certificates

Linux (Ubuntu/Debian)

Copy certificates to system certificate directory:

# Copy certificates to the ca-certificates directory
sudo cp ./opentdf-certs/keycloak.opentdf.local.crt /usr/local/share/ca-certificates/
sudo cp ./opentdf-certs/platform.opentdf.local.crt /usr/local/share/ca-certificates/

# Update the certificate store
sudo update-ca-certificates

Verify certificates are installed:

# Check if certificates are in the trusted store
awk -v cmd='openssl x509 -noout -subject' '/BEGIN/{close(cmd)};{print | cmd}' < /etc/ssl/certs/ca-certificates.crt | grep opentdf

Linux (RHEL/CentOS/Fedora)

Copy certificates to system certificate directory:

# Copy certificates to the ca-trust directory
sudo cp ./opentdf-certs/keycloak.opentdf.local.crt /etc/pki/ca-trust/source/anchors/
sudo cp ./opentdf-certs/platform.opentdf.local.crt /etc/pki/ca-trust/source/anchors/

# Update the certificate store
sudo update-ca-trust

Verify certificates are installed:

# Check if certificates are trusted
trust list | grep "opentdf.local"

Linux (Arch Linux)

Copy certificates to system certificate directory:

# Copy certificates to the ca-certificates directory
sudo cp ./opentdf-certs/keycloak.opentdf.local.crt /etc/ca-certificates/trust-source/anchors/
sudo cp ./opentdf-certs/platform.opentdf.local.crt /etc/ca-certificates/trust-source/anchors/

# Update the certificate store
sudo trust extract-compat

Browser-specific Certificate Installation

Some browsers (like Firefox) maintain their own certificate stores. If you continue to see certificate warnings in your browser after following the OS-specific steps above, you may need to import the certificates directly into your browser's certificate store.

Certificate Validity

These self-signed certificates are only valid for local development. Never use self-signed certificates in production environments.

Create a Profile

Creating a profile allows us to store the host we want to connect to and then to tie our credentials to it when we login.

otdfctl profile create platform-otdf-local https://platform.opentdf.local:8443

During the provisioning of the platform, a few test clients were created in Keycloak. You can use the following credentials to login to the platform. This will cache those credentials locally for the otdfctl CLI. You can also pass in the flags --with-client-creds and --with-client-creds-file on every command to authenticate with the platform.

otdfctl --profile platform-otdf-local auth client-credentials opentdf secret

Create A Namespace

In this first step you are going to create a new namespace. A namespace is a way to organize your attributes within the platform.

You will see an empty list of namespaces when you first start the platform.

otdfctl --profile platform-otdf-local policy attributes namespaces list

SUCCESS   Found namespaces list
┌─────────────────────────┬───────────────────────────┬─────────────────────────────┬────────────────────────────┬────────────────────────────────┬────────────────────────────────┐
│Id                       │Name                       │Active                       │Labels                      │Created At                      │Updated At                      │
├─────────────────────────┼───────────────────────────┼─────────────────────────────┼────────────────────────────┼────────────────────────────────┼────────────────────────────────┤
└─────────────────────────┴───────────────────────────┴─────────────────────────────┴────────────────────────────┴────────────────────────────────┴────────────────────────────────┘
NOTE   Use 'namespaces get --id=<id> --json' to see all properties

Create a new namespace

otdfctl --profile platform-otdf-local policy attributes namespaces create --name opentdf.io

SUCCESS   Created namespaces: 7650f02a-be00-4faa-a1d1-37cded5e23dc
┌────────────────────────────────────────────────────────────────────────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────┐
│Property                                                                    │Value                                                                                                │
├────────────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────┤
│Name                                                                        │opentdf.io                                                                                           │
│Id                                                                          │7650f02a-be00-4faa-a1d1-37cded5e23dc                                                                 │
│Created At                                                                  │Mon Jun 24 11:02:00 UTC 2024                                                                         │
│Updated At                                                                  │Mon Jun 24 11:02:00 UTC 2024                                                                         │
└────────────────────────────────────────────────────────────────────────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────┘
NOTE   Use 'namespaces get --id=7650f02a-be00-4faa-a1d1-37cded5e23dc --json' to see all properties

List the namespaces

With that namespace created we should be able to see it in the list of namespaces as well as get the details of the namespace.

otdfctl --profile platform-otdf-local policy attributes namespaces list

SUCCESS   Found namespaces list
┌──────────────────────────────────────────────┬────────────────────┬────────────────┬────────────────┬──────────────────────────────────────┬─────────────────────────────────────┐
│Id                                            │Name                │Active          │Labels          │Created At                            │Updated At                           │
├──────────────────────────────────────────────┼────────────────────┼────────────────┼────────────────┼──────────────────────────────────────┼─────────────────────────────────────┤
│7650f02a-be00-4faa-a1d1-37cded5e23dc          │opentdf.io          │true            │[]              │Mon Jun 24 11:02:00 UTC 2024          │Mon Jun 24 11:02:00 UTC 2024         │
└──────────────────────────────────────────────┴────────────────────┴────────────────┴────────────────┴──────────────────────────────────────┴─────────────────────────────────────┘
NOTE   Use 'namespaces get --id=<id> --json' to see all properties

Let's export the namespace id for later use.

export NAMESPACE_ID=<id>

Get the details of the namespace

otdfctl --profile platform-otdf-local policy attributes namespaces get --id=$NAMESPACE_ID

SUCCESS   Found namespaces: 7650f02a-be00-4faa-a1d1-37cded5e23dc
┌────────────────────────────────────────────────────────────────────────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────┐
│Property                                                                    │Value                                                                                                │
├────────────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────┤
│Id                                                                          │7650f02a-be00-4faa-a1d1-37cded5e23dc                                                                 │
│Name                                                                        │opentdf.io                                                                                           │
│Created At                                                                  │Mon Jun 24 11:02:00 UTC 2024                                                                         │
│Updated At                                                                  │Mon Jun 24 11:02:00 UTC 2024                                                                         │
└────────────────────────────────────────────────────────────────────────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────┘
NOTE   Use 'namespaces get --id=7650f02a-be00-4faa-a1d1-37cded5e23dc --json' to see all properties

Create an Attribute

Now that we have a namespace we can create a new attribute. An attribute is a way to define a classification for your data.

It consists of two main parts:

A definition which describes how an entities entitlements and resource attributes are compared.
1. ANY_OF: The entity must have at least one of the values in the list. This is your typical OR operation.
2. ALL_OF: The entity must have all of the values in the list. This is your typical AND operation.
3. HIERARCHY: This is an ordered list of values. The entity must have an equal or greater value in the list.
A list of values

warning

Attribute name and values are case insensitive. This means ROLE is equal to role.

tip

You can also create values at the same time when creating the definition. This is useful when you know the values you want to use. (e.g. --value admin --value developer --value guest)

Create attribute definition

otdfctl --profile platform-otdf-local policy attributes create --name role  -s $NAMESPACE_ID  -r ANY_OF

SUCCESS   Created attributes: bd02d7ab-564d-4b6c-95c4-3d4a8a259000
┌────────────────────────────────────────────────────────────────────────────────┬─────────────────────────────────────────────────────────────────────────────────────────────────┐
│Property                                                                        │Value                                                                                            │
├────────────────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────┤
│Name                                                                            │role                                                                                             │
│Rule                                                                            │ANY_OF                                                                                           │
│Values                                                                          │[]                                                                                               │
│Namespace                                                                       │opentdf.io                                                                                       │
│Created At                                                                      │Mon Jun 24 11:09:39 UTC 2024                                                                     │
│Updated At                                                                      │Mon Jun 24 11:09:39 UTC 2024                                                                     │
└────────────────────────────────────────────────────────────────────────────────┴─────────────────────────────────────────────────────────────────────────────────────────────────┘
NOTE   Use 'attributes get --id=bd02d7ab-564d-4b6c-95c4-3d4a8a259000 --json' to see all properties

Let's export the attribute id for later use.

export ATTRIBUTE_ID=<id>

Create attribute values

# Create admin value
otdfctl --profile platform-otdf-local policy attributes values create -a $ATTRIBUTE_ID --value admin

SUCCESS   Created values: 0fe7e8d0-a3ff-485f-ac24-a54d85904712
┌──────────────────────────────────────────────────────────────────────────┬───────────────────────────────────────────────────────────────────────────────────────────────────────┐
│Property                                                                  │Value                                                                                                  │
├──────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────┤
│Id                                                                        │0fe7e8d0-a3ff-485f-ac24-a54d85904712                                                                   │
│FQN                                                                       │https://opentdf.io/attr/role/value/admin                                                               │
│Value                                                                     │admin                                                                                                  │
│Created At                                                                │Mon Jun 24 11:11:15 UTC 2024                                                                           │
│Updated At                                                                │Mon Jun 24 11:11:15 UTC 2024                                                                           │
└──────────────────────────────────────────────────────────────────────────┴───────────────────────────────────────────────────────────────────────────────────────────────────────┘
NOTE   Use 'values get --id=0fe7e8d0-a3ff-485f-ac24-a54d85904712 --json' to see all properties

Export the admin value id for later use.

export ADMIN_VALUE_ID=<id>

# Create developer value
otdfctl --profile platform-otdf-local policy attributes values create -a $ATTRIBUTE_ID --value developer

SUCCESS   Created values: dbfcbe15-7392-4e35-9e1d-3d06918472be
┌────────────────────────────────────────────────────────────────────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│Property                                                                │Value                                                                                                    │
├────────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│Id                                                                      │dbfcbe15-7392-4e35-9e1d-3d06918472be                                                                     │
│FQN                                                                     │https://opentdf.io/attr/role/value/developer                                                             │
│Value                                                                   │developer                                                                                                │
│Created At                                                              │Mon Jun 24 11:12:45 UTC 2024                                                                             │
│Updated At                                                              │Mon Jun 24 11:12:45 UTC 2024                                                                             │
└────────────────────────────────────────────────────────────────────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────┘
NOTE   Use 'values get --id=dbfcbe15-7392-4e35-9e1d-3d06918472be --json' to see all properties

Export the developer value id for later use.

export DEVELOPER_VALUE_ID=<id>

# Create guest value
otdfctl --profile platform-otdf-local policy attributes values create -a $ATTRIBUTE_ID --value guest

SUCCESS   Created values: 654f0877-2c0b-4a62-a9c3-87ed42bf77ac
┌──────────────────────────────────────────────────────────────────────────┬───────────────────────────────────────────────────────────────────────────────────────────────────────┐
│Property                                                                  │Value                                                                                                  │
├──────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────┤
│Id                                                                        │654f0877-2c0b-4a62-a9c3-87ed42bf77ac                                                                   │
│FQN                                                                       │https://opentdf.io/attr/role/value/guest                                                               │
│Value                                                                     │guest                                                                                                  │
│Created At                                                                │Mon Jun 24 11:14:22 UTC 2024                                                                           │
│Updated At                                                                │Mon Jun 24 11:14:22 UTC 2024                                                                           │
└──────────────────────────────────────────────────────────────────────────┴───────────────────────────────────────────────────────────────────────────────────────────────────────┘
NOTE   Use 'values get --id=654f0877-2c0b-4a62-a9c3-87ed42bf77ac --json' to see all properties

Export the guest value id for later use.

export GUEST_VALUE_ID=<id>

Get the attribute details

Now that we have listed values for the attribute, we can get the details of the attribute. You should see the added values in the list now.

otdfctl --profile platform-otdf-local policy attributes get --id=$ATTRIBUTE_ID

SUCCESS   Found attributes: bd02d7ab-564d-4b6c-95c4-3d4a8a259000
┌────────────────────────────────────────────────────────────────────────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────┐
│Property                                                                    │Value                                                                                                │
├────────────────────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────┤
│Id                                                                          │bd02d7ab-564d-4b6c-95c4-3d4a8a259000                                                                 │
│Name                                                                        │role                                                                                                 │
│Rule                                                                        │ANY_OF                                                                                               │
│Values                                                                      │[admin, developer, guest]                                                                            │
│Namespace                                                                   │opentdf.io                                                                                           │
│Created At                                                                  │Mon Jun 24 11:09:39 UTC 2024                                                                         │
│Updated At                                                                  │Mon Jun 24 11:14:22 UTC 2024                                                                         │
└────────────────────────────────────────────────────────────────────────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────┘
NOTE   Use 'attributes get --id=bd02d7ab-564d-4b6c-95c4-3d4a8a259000 --json' to see all properties

Create a Subject Mapping

A subject mapping is a way to define how a user or entity is mapped to an attribute in the OpenTDF Platform.

An attribute that is mapped to an entity is considered an entitlement. An entitlement is a way to define what tdf resources an entity can access.

With subject mappings there is a sub-resource called subject condition sets. These are a set of conditions that must be met for the subject mapping to be applied.

Create a Subject Condition Set

Download the example subject condition set. You can use this file to create a new condition set.

 otdfctl --profile platform-otdf-local policy subject-condition-sets create -j <path to file>/subject_condition_set.json

SUCCESS   Created subject-condition-sets: 74bf521f-5a79-48fe-acb8-b4b63ee7950b
┌──────────────┬───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│Property      │Value                                                                                                                                                              │
├──────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│Id            │74bf521f-5a79-48fe-acb8-b4b63ee7950b                                                                                                                               │
│SubjectSets   │[{"condition_groups":[{"conditions":[{"subject_external_selector_value":".clientId","operator":1,"subject_external_values":["opentdf"]}],"boolean_operator":1}]}]  │
│Created At    │Mon Jun 24 11:33:28 UTC 2024                                                                                                                                       │
│Updated At    │Mon Jun 24 11:33:28 UTC 2024                                                                                                                                       │
└──────────────┴───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
NOTE   Use 'subject-condition-sets get --id=74bf521f-5a79-48fe-acb8-b4b63ee7950b --json' to see all properties

Let's export the subject condition set id for later use.

export SUBJECT_CONDITION_SET_ID=<id>

Create a Subject Mapping

With the condition set created we can now create a new subject mapping that is tied to the subject condition set.

otdfctl --profile platform-otdf-local policy subject-mappings create --action read --attribute-value-id $DEVELOPER_VALUE_ID --subject-condition-set-id $SUBJECT_CONDITION_SET_ID

SUCCESS   Created subject-mappings: 751054f8-14da-44c0-9341-5dab36b8256d
┌─────────────────────────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│Property                 │Value                                                                                                                                                   │
├─────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│Id                       │751054f8-14da-44c0-9341-5dab36b8256d                                                                                                                    │
│Subject AttrVal: Id      │dbfcbe15-7392-4e35-9e1d-3d06918472be                                                                                                                    │
│Actions                  │[{"Value":{"Standard":1}}]                                                                                                                              │
│Subject Condition Set: I…│74bf521f-5a79-48fe-acb8-b4b63ee7950b                                                                                                                    │
│Subject Condition Set    │[{"condition_groups":[{"conditions":[{"subject_external_selector_value":".clientId","operator":1,"subject_external_values":["opentdf"]}],"boolean_opera…│
│Attribute Value Id       │dbfcbe15-7392-4e35-9e1d-3d06918472be                                                                                                                    │
│Created At               │Mon Jun 24 11:35:13 UTC 2024                                                                                                                            │
│Updated At               │Mon Jun 24 11:35:13 UTC 2024                                                                                                                            │
└─────────────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
NOTE   Use 'subject-mappings get --id=751054f8-14da-44c0-9341-5dab36b8256d --json' to see all properties

Let's export the subject mapping id for later use.

export SUBJECT_MAPPING_ID=<id>

Encrypt Some Data With TDF

Now that we have a few basic resources in place we can show you how to control access to data using the OpenTDF Platform.

Encrypt Data Without Attributes

Within the otdfctl CLI there is basic functionality to interact with TDF.

Example TDF encryption

echo 'my first encrypted tdf' | otdfctl encrypt --profile platform-otdf-local -o example.tdf --tdf-type tdf3

Decrypt Data Without Attributes

Because we didn't add any attributes to the data we encrypted, we should be able to decrypt the data without any issues.

otdfctl decrypt --profile platform-otdf-local --tdf-type tdf3 example.tdf

# Output
my first encrypted tdf

Encrypt Data With Attributes

In this example we will encrypt the data with the attribute https://opentdf.io/attr/role/value/guest. First cleanup any existing tdf files from before.

rm example.tdf

Example TDF encryption with attributes

echo 'my first encrypted tdf' | otdfctl encrypt --profile platform-otdf-local -o example.tdf --tdf-type tdf3 --attr https://opentdf.io/attr/role/value/guest

Decrypt Data With Attributes

In this first example we will try to decrypt the data but it will fail because we shouldn't be assigned the entitlement of https://opentdf.io/attr/role/value/guest at this point.

Example TDF failed decryption

otdfctl decrypt --profile platform-otdf-local --tdf-type tdf3 example.tdf

# Output
ERROR    Failed to decrypt file: reader.WriteTo failed: doPayloadKeyUnwrap splitKey.rewrap failed: error making request to kas: error making rewrap request: rpc error: code = PermissionDenied desc = request error
            rpc error: code = PermissionDenied desc = forbidden

What we have to do now is assign the entitlement of https://opentdf.io/attr/role/value/guest to the entity by creating a new subject mapping for the condition set we created earlier.

Create a new subject mapping

otdfctl --profile platform-otdf-local policy subject-mappings create --action read --attribute-value-id $GUEST_VALUE_ID --subject-condition-set-id $SUBJECT_CONDITION_SET_ID

SUCCESS   Created subject-mappings: f6bf9fc7-e23b-4276-9dfc-7a61f359edfd
┌─────────────────────────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│Property                 │Value                                                                                                                                                   │
├─────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│Id                       │f6bf9fc7-e23b-4276-9dfc-7a61f359edfd                                                                                                                    │
│Subject AttrVal: Id      │654f0877-2c0b-4a62-a9c3-87ed42bf77ac                                                                                                                    │
│Actions                  │[{"Value":{"Standard":1}}]                                                                                                                              │
│Subject Condition Set: I…│74bf521f-5a79-48fe-acb8-b4b63ee7950b                                                                                                                    │
│Subject Condition Set    │[{"condition_groups":[{"conditions":[{"subject_external_selector_value":".clientId","operator":1,"subject_external_values":["opentdf"]}],"boolean_opera…│
│Attribute Value Id       │654f0877-2c0b-4a62-a9c3-87ed42bf77ac                                                                                                                    │
│Created At               │Mon Jun 24 13:28:22 UTC 2024                                                                                                                            │
│Updated At               │Mon Jun 24 13:28:22 UTC 2024                                                                                                                            │
└─────────────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
NOTE   Use 'subject-mappings get --id=f6bf9fc7-e23b-4276-9dfc-7a61f359edfd --json' to see all properties

Now that we have the subject mapping in place we should be able to decrypt the data.

Example TDF successful decryption

otdfctl decrypt --profile platform-otdf-local --tdf-type tdf3 example.tdf

# Output
my first encrypted tdf

Takeaways and Next Steps

In this document you have learned how to create a namespace, attribute, subject mapping, and encrypt/decrypt data with the OpenTDF Platform.

Pre-requisites​

Update /etc/hosts​

Getting the Platform Running​

Save the Docker Compose File​

Start the Platform​

Trust Self Signed Certificates​

Import and Trust Certificates by Operating System​

Create a Profile​

Login to the Platform​

Create A Namespace​

Create a new namespace​

List the namespaces​

Get the details of the namespace​

Create an Attribute​

Create attribute definition​

Create attribute values​

Get the attribute details​

Create a Subject Mapping​

Create a Subject Condition Set​

Create a Subject Mapping​

Encrypt Some Data With TDF​

Encrypt Data Without Attributes​

Example TDF encryption​

Decrypt Data Without Attributes​

Encrypt Data With Attributes​

Example TDF encryption with attributes​

Decrypt Data With Attributes​

Example TDF failed decryption​

Create a new subject mapping​

Example TDF successful decryption​

Takeaways and Next Steps​

Pre-requisites

Update /etc/hosts

Getting the Platform Running

Save the Docker Compose File

Start the Platform

Trust Self Signed Certificates

Import and Trust Certificates by Operating System

Create a Profile

Login to the Platform

Create A Namespace

Create a new namespace

List the namespaces

Get the details of the namespace

Create an Attribute

Create attribute definition

Create attribute values

Get the attribute details

Create a Subject Mapping

Create a Subject Condition Set

Create a Subject Mapping

Encrypt Some Data With TDF

Encrypt Data Without Attributes

Example TDF encryption

Decrypt Data Without Attributes

Encrypt Data With Attributes

Example TDF encryption with attributes

Decrypt Data With Attributes

Example TDF failed decryption

Create a new subject mapping

Example TDF successful decryption

Takeaways and Next Steps