Migration Guide#

This guide is for system administrators, and will help you migrate your NexusLIMS installation from v1.4.3 to v2.5.0.

What Changed#

The major changes you need to know about:

  1. Package Manager: Poetry → uv (faster, simpler installation)

  2. Environment Variables: All variables now use NX_ prefix

  3. Metadata System: New Pydantic schemas with EM Glossary integration and validation

  4. Configuration: Better validation with helpful error messages

Transition to uv#

The project has transitioned from Poetry to uv for Python package and environment management. uv is a fast, modern package installer and resolver written in Rust that provides:

  • Significantly faster dependency resolution and installation

  • Better reproducibility with lock files

  • Simpler workflow for both development and deployment

  • Direct Python version management (no need for separate tools like pyenv)

Installing uv#

Install uv using the official installation script:

# Unix/macOS/Linux
curl -LsSf https://astral.sh/uv/install.sh | sh

# Windows
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"

For alternative installation methods, see the uv documentation.

Environment Variable Changes#

Environment variable naming has been standardized for better consistency. You must update your .env file to use the new variable names.

Renamed Variables#

The following variables have been renamed:

Old Name (v1.x)

New Name (v2.0+)

Required

mmfnexus_path

NX_INSTRUMENT_DATA_PATH

Yes

nexusLIMS_path

NX_DATA_PATH

Yes

nexusLIMS_db_path

NX_DB_PATH

Yes

nexusLIMS_user

NX_CDCS_TOKEN

Yes

nexusLIMS_pass

NX_CDCS_TOKEN

Yes

cdcs_url

NX_CDCS_URL

Yes

NexusLIMS_cert_bundle_file

NX_CERT_BUNDLE_FILE

No

NexusLIMS_cert_bundle

NX_CERT_BUNDLE

No

NexusLIMS_file_strategy

NX_FILE_STRATEGY

No

NexusLIMS_ignore_patterns

NX_IGNORE_PATTERNS

No

nexusLIMS_file_delay_days

NX_FILE_DELAY_DAYS

No

NEMO_address_*

NX_NEMO_ADDRESS_*

If using NEMO

NEMO_token_*

NX_NEMO_TOKEN_*

If using NEMO

NEMO_strftime_fmt_*

NX_NEMO_STRFTIME_FMT_*

No

NEMO_strptime_fmt_*

NX_NEMO_STRPTIME_FMT_*

No

NEMO_tz_*

NX_NEMO_TZ_*

No

Note

v2.3.0+ Authentication Change: CDCS authentication has changed from username/password (NX_CDCS_USER and NX_CDCS_PASS) to token-based authentication (NX_CDCS_TOKEN). Replace your username and password with an API token obtained from the CDCS admin panel. See NX_CDCS_TOKEN for details.

New Variables#

The following variables are new in v2.0:

Variable

Purpose

Required

NX_LOG_PATH

Custom directory for application logs

No

NX_RECORDS_PATH

Custom directory for generated XML records

No

NX_LOCAL_PROFILES_PATH

Directory for site-specific instrument profiles

No

NX_EMAIL_SMTP_HOST

SMTP server hostname

If using email

NX_EMAIL_SMTP_PORT

SMTP server port (default: 587)

No

NX_EMAIL_SMTP_USERNAME

SMTP username for authentication

No

NX_EMAIL_SMTP_PASSWORD

SMTP password for authentication

No

NX_EMAIL_USE_TLS

Use TLS encryption (default: true)

No

NX_EMAIL_SENDER

Email address to send from

If using email

NX_EMAIL_RECIPIENTS

Comma-separated recipient addresses

If using email

Removed Variables#

The following variables from v1.x are no longer used:

Variable

Reason for Removal

test_cdcs_url

Tests now run as independent unit tests and integration tests against a Docker-deployed instance created on demand

sharepoint_root_url

SharePoint harvester has been removed

email_sender

Replaced by enhanced email configuration (NX_EMAIL_*)

email_recipients

Replaced by enhanced email configuration (NX_EMAIL_*)

Complete Environment Variable Reference#

General Configuration#

Variable

Description

Example

NX_FILE_STRATEGY

File inclusion strategy: exclusive (only files with extractors) or inclusive (all files)

exclusive

NX_IGNORE_PATTERNS

JSON array of glob patterns to ignore

["*.mib","*.db","*.emi"]

NX_INSTRUMENT_DATA_PATH

Root path to centralized instrument data (read-only mount)

/mnt/microscopy/data

NX_DATA_PATH

Writable parallel path for metadata and previews

/var/nexuslims/data

NX_DB_PATH

Path to NexusLIMS SQLite database

/var/nexuslims/nexuslims.db

NX_FILE_DELAY_DAYS

Maximum days to wait for files (can be fractional)

2

NX_LOG_PATH

Optional: Directory for logs (defaults to NX_DATA_PATH/logs/)

/var/log/nexuslims

NX_RECORDS_PATH

Optional: Directory for records (defaults to NX_DATA_PATH/records/)

/var/nexuslims/records

NX_LOCAL_PROFILES_PATH

Optional: Directory for site-specific instrument profiles

/var/nexuslims/profiles

CDCS Authentication#

Variable

Description

Example

NX_CDCS_TOKEN

API token for CDCS authentication

your-api-token-here

NX_CDCS_URL

Root URL of NexusLIMS CDCS instance (with trailing slash)

https://nexuslims.example.com/

NX_CERT_BUNDLE_FILE

Optional: Path to SSL certificate bundle

/etc/ssl/certs/custom-ca.pem

NX_CERT_BUNDLE

Optional: SSL certificate bundle as string

-----BEGIN CERTIFICATE-----\n...

NEMO Harvester Configuration#

Multiple NEMO harvesters can be configured using numbered suffixes (_1, _2, etc.):

Variable

Description

Example

NX_NEMO_ADDRESS_N

Full path to NEMO API root (with trailing slash)

https://nemo.example.com/api/

NX_NEMO_TOKEN_N

Authentication token for NEMO server

abc123def456...

NX_NEMO_STRFTIME_FMT_N

Optional: Format for sending datetimes to API

%Y-%m-%dT%H:%M:%S%z

NX_NEMO_STRPTIME_FMT_N

Optional: Format for parsing datetimes from API

%Y-%m-%dT%H:%M:%S%z

NX_NEMO_TZ_N

Optional: IANA timezone for API datetimes

America/Denver

Email Notification Configuration#

Email notifications are sent by nexuslims build-records when errors are detected:

Variable

Description

Example

NX_EMAIL_SMTP_HOST

SMTP server hostname

smtp.gmail.com

NX_EMAIL_SMTP_PORT

Optional: SMTP port (default: 587)

587

NX_EMAIL_SMTP_USERNAME

Optional: SMTP username for authentication

user@example.com

NX_EMAIL_SMTP_PASSWORD

Optional: SMTP password for authentication

your-app-password

NX_EMAIL_USE_TLS

Optional: Use TLS encryption (default: true)

true

NX_EMAIL_SENDER

Email address to send from

nexuslims@example.com

NX_EMAIL_RECIPIENTS

Comma-separated list of recipient addresses

admin@example.com,team@example.com

Centralized Configuration Module#

To improve maintainability and type safety, all environment variable access is centralized in the nexusLIMS.config module using pydantic-settings.

For complete configuration documentation, see Configuration.

What This Means for Users#

  • No file structure changes required: Continue defining variables in your .env file or system environment

  • Automatic validation: The application validates configuration at startup and provides clear error messages

  • Type safety: Configuration values are automatically converted to appropriate Python types

  • Default values: Sensible defaults are provided where applicable

  • Mocking for tests: The design of the system allows for a fully mocked setup for use in the test suite, ensuring more reliable code quality testing.

What This Means for Developers#

Instead of accessing environment variables directly:

# Old approach (v1.x) - Don't use
import os
path = os.environ.get("nexusLIMS_path")

Use the centralized config module:

# New approach (v2.0+)
from nexusLIMS.config import settings
path = settings.NX_DATA_PATH

This provides:

  • Type hints and autocomplete in IDEs

  • Validation at import time

  • Consistent defaults throughout the application

  • Easier testing with refresh_settings()

Step-by-Step Migration Instructions#

Follow these steps to migrate your NexusLIMS installation from v1.x to v2.5.0:

1. Backup Your Current Installation#

Important

Database migrations required: The database schema has changed since v1.4.3. You will need to run migrations to:

  • Add the upload_log table (for tracking record exports)

  • Update session_log CHECK constraints (to support new record statuses)

Note: Database migrations automatically create timestamped backups before making changes. However, it’s still good practice to create your own backups of critical files before major upgrades.

# Backup your .env file (always recommended)
cp .env .env.v1.backup

# Optional: Create manual database backup (migrations create automatic backups)
cp /path/to/nexuslims_db.sqlite /path/to/nexuslims_db.sqlite.backup

2. Install uv#

curl -LsSf https://astral.sh/uv/install.sh | sh

# Verify installation
uv --version

3. Update Your .env File#

Create a new .env file based on .env.example:

# Copy the example file
cp .env.example .env

# Edit with your settings
nano .env  # or your preferred editor

Map your old variables to new names using the table in the “Renamed Variables” section above.

Key changes to make:

  • Rename mmfnexus_path → NX_INSTRUMENT_DATA_PATH

  • Rename nexusLIMS_path → NX_DATA_PATH

  • Rename nexusLIMS_db_path → NX_DB_PATH

  • Replace nexusLIMS_user and nexusLIMS_pass → NX_CDCS_TOKEN (obtain token from CDCS admin panel)

  • Rename cdcs_url → NX_CDCS_URL

  • Rename NexusLIMS_cert_bundle_file → NX_CERT_BUNDLE_FILE (if used)

  • Rename NexusLIMS_cert_bundle → NX_CERT_BUNDLE (if used)

  • Rename NexusLIMS_file_strategy → NX_FILE_STRATEGY (if customized)

  • Rename NexusLIMS_ignore_patterns → NX_IGNORE_PATTERNS (if customized)

  • Rename nexusLIMS_file_delay_days → NX_FILE_DELAY_DAYS (if customized)

  • Rename NEMO_address_* → NX_NEMO_ADDRESS_* (if using NEMO)

  • Rename NEMO_token_* → NX_NEMO_TOKEN_* (if using NEMO)

  • Remove test_cdcs_url, sharepoint_root_url, email_sender, email_recipients

  • Add new email configuration variables (NX_EMAIL_*) if you want email notifications

4. Remove Old Virtual Environment#

# If you have an old Poetry or pyenv environment
rm -rf .venv

# Remove Poetry lock file
rm poetry.lock  # if it exists

5. Create New Environment with uv#

# Install NexusLIMS with dependencies (this will create the virtualenvironment as necessary)
uv sync

6. Mark Database as Migrated and Apply Updates#

Added in version 2.2.0: NexusLIMS now uses Alembic for database schema version control. You need to mark your existing database as migrated to the baseline schema, then apply updates.

# Mark database as at the baseline schema version (v1.4.3 schema)
nexuslims db alembic stamp v1_4_3

# Apply pending migrations (adds upload_log table and updated CHECK constraints)
nexuslims db upgrade

What this does:

  1. stamp v1_4_3 - Marks your v1.4.3 database as having the baseline schema (instruments + session_log tables)

  2. upgrade - Applies migration (database update) commands that have updated the schema since v1.4.3:

    • Migration v2_4_0a: Adds the upload_log table for tracking record exports

    • Migration v2_4_0b: Updates session_log CHECK constraints to include new statuses (NO_CONSENT, NO_RESERVATION, BUILT_NOT_EXPORTED)

Note

If you’re running from a source checkout (not an installed package), prefix commands with uv run:

uv run nexuslims db alembic stamp v1_4_3
uv run nexuslims db upgrade

Note

Automatic backups: The migration system automatically creates timestamped backups before making changes to your database (e.g., database_backup_20260207_143216.sqlite). These are created in the same directory as your database file.

If you prefer to create your own backup first, you can:

cp /path/to/nexuslims_db.sqlite /path/to/nexuslims_db.sqlite.backup

7. Verify Installation#

# Test configuration loading
uv run python -c "from nexusLIMS.config import settings; print(settings.NX_DATA_PATH)"

# Run unit tests (not necessary, but will provide confidence things are working)
uv run pytest tests/unit

# Check database connection and list instruments
uv run python -c "from nexusLIMS.instruments import instrument_db; print(f'{len(instrument_db)} instruments found:'); [print(f'  - {pid}') for pid in instrument_db.keys()]"

8. Update Deployment Scripts#

If you have cron jobs or systemd services running NexusLIMS:

Old command style (v1.x):

# Using Poetry
cd /path/to/NexusLIMS
poetry run python -m nexusLIMS.builder.record_builder

New command style (v2.0+):

# Using uv
cd /path/to/NexusLIMS
uv run nexuslims build-records

9. Test Record Building#

Run a test record build to verify everything works:

# Dry run mode (find files but don't build records)
uv run nexuslims build-records -n

# Verbose mode (with detailed logging)
uv run nexuslims build-records -vv -n

Common Migration Issues#

Database Path Not Found#

Error: ValidationError: NX_DB_PATH ... does not exist

Solution: Ensure your database file exists at the path specified in NX_DB_PATH. If you migrated the path, update it in your .env file.

Directory Does Not Exist#

Error: ValidationError: NX_INSTRUMENT_DATA_PATH ... does not exist

Solution: Pydantic validates that paths exist. Ensure all directory paths in your .env file point to existing directories:

mkdir -p /path/to/nexuslims/data
mkdir -p /path/to/nexuslims/logs

SSL Certificate Errors#

Error: SSLError: certificate verify failed

Solution: If your CDCS or NEMO servers use custom SSL certificates, configure NX_CERT_BUNDLE_FILE:

NX_CERT_BUNDLE_FILE=/etc/ssl/certs/custom-ca-bundle.pem

NEMO Harvester Not Found#

Error: No NEMO reservations being harvested

Solution: Verify your NEMO configuration uses the new NX_ prefix:

NX_NEMO_ADDRESS_1=https://nemo.example.com/api/
NX_NEMO_TOKEN_1=your-token-here

Getting Help#

If you encounter issues during migration:

  1. Review the error message carefully - Pydantic provides detailed validation errors

  2. Check the example file: Compare your .env with .env.example

  3. Consult the logs: Check NX_LOG_PATH (or NX_DATA_PATH/logs/) for details

  4. Open an issue: datasophos/NexusLIMS#issues

For professional migration support, deployment assistance, or custom development:

  • Contact Datasophos: https://datasophos.co/#contact

Further Reading#