docs: add virtual-schemas specification #98

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

dimitri-yatsenko merged 3 commits into pre/v2.0 from virtual-modules

Jan 9, 2026

src/how-to/configure-database.md

-Original file line number
+Diff line change
@@ Expand Up / @@ -122,3 +122,60 @@ For local development without TLS: @@
     }
     ```
+    ## Connection Lifecycle
+    ### Persistent Connection (Default)
+    DataJoint uses a persistent singleton connection by default:
+    ```python
+    import datajoint as dj
+    # First call establishes connection
+    conn = dj.conn()
+    # Subsequent calls return the same connection
+    conn2 = dj.conn()  # Same as conn
+    # Reset to create a new connection
+    conn3 = dj.conn(reset=True)  # New connection
+    ```
+    This is ideal for interactive sessions and notebooks.
+    ### Context Manager (Explicit Cleanup)
+    For serverless environments (AWS Lambda, Cloud Functions) or when you need explicit connection lifecycle control, use the context manager:
+    ```python
+    import datajoint as dj
+    with dj.Connection(host, user, password) as conn:
+        schema = dj.schema('my_schema', connection=conn)
+        MyTable().insert(data)
+    # Connection automatically closed when exiting the block
+    ```
+    The connection closes automatically even if an exception occurs:
+    ```python
+    try:
+        with dj.Connection(**creds) as conn:
+            schema = dj.schema('my_schema', connection=conn)
+            MyTable().insert(data)
+            raise SomeError()
+    except SomeError:
+        pass
+    # Connection is still closed properly
+    ```
+    ### Manual Close
+    You can also close a connection explicitly:
+    ```python
+    conn = dj.conn()
+    # ... do work ...
+    conn.close()
+    ```

src/how-to/index.md

-Original file line number
+Diff line change
@@ Expand Up @@
     - [Installation](installation.md) — Installing DataJoint
     - [Configure Database Connection](configure-database.md) — Connection settings
     - [Configure Object Storage](configure-storage.md) — S3, MinIO, file stores
+    - [Use the Command-Line Interface](use-cli.md) — Interactive REPL
     ## Schema Design
@@ Expand Down @@

src/how-to/use-cli.md

-Original file line number
+Diff line change
@@ -0,0 +1,138 @@
+    # Use the Command-Line Interface
+    Start an interactive Python REPL with DataJoint pre-loaded.
+    The `dj` command provides quick access to DataJoint for exploring schemas, running queries, and testing connections without writing scripts.
+    ## Start the REPL
+    ```bash
+    dj
+    ```
+    This opens a Python REPL with `dj` (DataJoint) already imported:
+    ```
+    DataJoint 2.0.0 REPL
+    Type 'dj.' and press Tab for available functions.
+    >>> dj.conn()  # Connect to database
+    >>> dj.list_schemas()  # List available schemas
+    ```
+    ## Specify Database Credentials
+    Override config file settings from the command line:
+    ```bash
+    dj --host localhost:3306 --user root --password secret
+    ```
+    | Option | Description |
+    |--------|-------------|
+    | `--host HOST` | Database host as `host:port` |
+    | `-u`, `--user USER` | Database username |
+    | `-p`, `--password PASS` | Database password |
+    Credentials from command-line arguments override values in config files.
+    ## Load Schemas as Virtual Modules
+    Load database schemas directly into the REPL namespace:
+    ```bash
+    dj -s my_lab:lab -s my_analysis:analysis
+    ```
+    The format is `schema_name:alias` where:
+    - `schema_name` is the database schema name
+    - `alias` is the variable name in the REPL
+    This outputs:
+    ```
+    DataJoint 2.0.0 REPL
+    Type 'dj.' and press Tab for available functions.
+    Loaded schemas:
+      lab -> my_lab
+      analysis -> my_analysis
+    >>> lab.Subject.to_dicts()  # Query Subject table
+    >>> dj.Diagram(lab.schema)  # View schema diagram
+    ```
+    ## Common Workflows
+    ### Explore an Existing Schema
+    ```bash
+    dj -s production_db:db
+    ```
+    ```python
+    >>> list(db.schema)  # List all tables
+    >>> db.Experiment().to_dicts()[:5]  # Preview data
+    >>> dj.Diagram(db.schema)  # Visualize structure
+    ```
+    ### Quick Data Check
+    ```bash
+    dj --host db.example.com -s my_lab:lab
+    ```
+    ```python
+    >>> len(lab.Session())  # Count sessions
+    >>> lab.Session.describe()  # Show table definition
+    ```
+    ### Test Connection
+    ```bash
+    dj --host localhost:3306 --user testuser --password testpass
+    ```
+    ```python
+    >>> dj.conn()  # Verify connection works
+    >>> dj.list_schemas()  # Check accessible schemas
+    ```
+    ## Version Information
+    Display DataJoint version:
+    ```bash
+    dj --version
+    ```
+    ## Help
+    Display all options:
+    ```bash
+    dj --help
+    ```
+    ## Entry Points
+    The CLI is available as both `dj` and `datajoint`:
+    ```bash
+    dj --version
+    datajoint --version  # Same command
+    ```
+    ## Programmatic Usage
+    The CLI function can also be called from Python:
+    ```python
+    from datajoint.cli import cli
+    # Show version and exit
+    cli(["--version"])
+    # Start REPL with schemas
+    cli(["-s", "my_lab:lab"])
+    ```

src/reference/specs/index.md

-Original file line number
+Diff line change
@@ Expand Up / @@ -29,6 +29,7 @@ Each specification follows a consistent structure: @@
     |---------------|-------------|
     | [Table Declaration](table-declaration.md) | Table definition syntax, tiers, foreign keys, and indexes |
     | [Master-Part Relationships](master-part.md) | Compositional data modeling, integrity, and cascading operations |
+    | [Virtual Schemas](virtual-schemas.md) | Accessing schemas without Python source, introspection API |
     ### Query Operations
@@ Expand Down @@

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs: add virtual-schemas specification #98

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!