(schema_documentation)=
# XML Schema Reference

This page provides automatically-generated documentation for the NexusLIMS XML schema used to represent experimental records.

## Introduction

NexusLIMS generates output experimental records as XML documents that conform to the [**Nexus Experiment Schema**](https://doi.org/10.18434/M32245) (XSD). These records can then be curated in the [NexusLIMS CDCS](https://github.com/datasophos/nexuslims-cdcs/) frontend application, and capture comprehensive metadata about microscopy sessions, including:

- **Session information**: Who performed the experiment, when, and why
- **Instrument details**: Which microscope(s) were used and their configurations
- **Acquisition activities**: Temporal grouping of related data files
- **Dataset metadata**: Comprehensive technical parameters for each file

### From Internal to Output XML Representations

```{versionadded} 2.2.0
Starting with v2.2.0, NexusLIMS uses a **two-stage metadata pipeline**. This validation and standardization system is still evolving as EM Glossary coverage expands and community standards develop.
```

1. **Extraction & Validation (Internal)**: NexusLIMS extractors produce metadata dictionaries validated against internal Pydantic schemas ({py:class}`~nexusLIMS.schemas.metadata.NexusMetadata`, {py:class}`~nexusLIMS.schemas.metadata.ImageMetadata`, {py:class}`~nexusLIMS.schemas.metadata.SpectrumMetadata`, etc.) with:
   - EM Glossary standardized fields (where available - the vocabulary is still developing)
   - Physical quantities with units (via Pint)
   - Basic validation and unit normalization

2. **Serialization to XML**: Validated metadata is transformed into XML following the [*Nexus Experiment Schema*](https://doi.org/10.18434/M32245) with:
   - Quantity fields → XML elements with `unit` attributes
   - Extension fields → `<summary>` elements in `<nx:extensions>` blocks
   - EM Glossary metadata → `emg:display_name` and `emg:id` attributes on quantity elements (where EMG terms are available)

#### Example Transformation

```python
# Pydantic metadata (Python objects)
{
    "acceleration_voltage": Quantity(200, "kV"),
    "magnification": Quantity(50, "kiloX"),
    "extensions": {
        "detector_brightness": 50.0,
    }
}
```

#### Metadata Serialized to XML (within a Dataset element)

```xml
<nx:dataset type="Image" role="Experimental">
    <nx:name>micrograph_001.tif</nx:name>
    <nx:location>/path/to/data/micrograph_001.tif</nx:location>
    <nx:format>image/tiff</nx:format>
    <nx:meta name="Acceleration Voltage" unit="kV">200.0</nx:meta>
    <nx:meta name="Magnification" unit="kX">50</nx:meta>
    <nx:meta name="detector_brightness">50.0</nx:meta>
</nx:dataset>
```

This two-stage approach provides:

- **Basic type safety** during extraction (Pydantic validation)
- **Emerging standardization** via EM Glossary (where terms are available)
- **Interoperability** through standard XML format
- **Semantic enrichment** with EMG metadata attributes (where EMG IDs are available)

See {doc}`nexuslims_internal_schema` for complete details on the Pydantic schema system and {doc}`em_glossary_reference` for EM Glossary integration.

## Nexus Experiment Schema

The Nexus Experiment schema defines the final output structure of XML records generated by NexusLIMS. Each record describes a microscopy session, including metadata about the experiment, instruments used, and data files acquired.

```{eval-rst}
.. xsddoc:: ../nexusLIMS/schemas/nexus-experiment.xsd
```

## Example Record

Here is a simplified example of a complete NexusLIMS XML record showing the key structural elements:

```xml
<?xml version="1.0" encoding="UTF-8"?>
<nx:Experiment xmlns:nx="https://data.nist.gov/od/dm/nexus/experiment/v1.0"
               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xsi:schemaLocation="https://doi.org/10.18434/M32245"
               pid="record_2024-01-15_12345">

  <nx:title>STEM imaging session - nanoparticle characterization</nx:title>
  <nx:id>session_20240115_jsmith</nx:id>

  <!-- Session-level metadata -->
  <nx:summary ref="https://nemo.example.org/reservation/12345">
    <nx:experimenter>Dr. Jane Smith</nx:experimenter>
    <nx:experimenter>John Doe</nx:experimenter>
    <nx:instrument pid="FEI-Titan-TEM">FEI Titan TEM</nx:instrument>
    <nx:reservationStart>2024-01-15T10:00:00-05:00</nx:reservationStart>
    <nx:reservationEnd>2024-01-15T12:00:00-05:00</nx:reservationEnd>
    <nx:motivation>Characterize particle size distribution and elemental composition of Au-TiO2 nanoparticles</nx:motivation>
  </nx:summary>

  <!-- Sample description -->
  <nx:sample id="Au-TiO2-batch5" ref="https://sample-db.example.org/samples/Au-TiO2-batch5">
    <nx:name>Gold-Titania Nanoparticles Batch 5</nx:name>
    <nx:description>Au nanoparticles supported on TiO2, synthesized January 2024</nx:description>
    <nx:elements>
      <nx:Au/>
      <nx:Ti/>
      <nx:O/>
    </nx:elements>
  </nx:sample>

  <!-- Project information -->
  <nx:project>
    <nx:name>Nanoparticle Catalysis Study</nx:name>
    <nx:project_id>NP-2024-001</nx:project_id>
  </nx:project>

  <!-- Acquisition activity (temporal file grouping) -->
  <nx:acquisitionActivity seqno="1">
    <nx:startTime>2024-01-15T10:30:00-05:00</nx:startTime>
    <nx:sampleID>Au-TiO2-batch5</nx:sampleID>

    <!-- Setup parameters common to all datasets in this activity -->
    <nx:setup>
      <nx:param name="Microscope Mode">STEM</nx:param>
      <nx:param name="Detector">HAADF</nx:param>
    </nx:setup>

    <!-- Individual dataset -->
    <nx:dataset type="Image" role="Experimental">
      <nx:name>overview_001.dm3</nx:name>
      <nx:location>2024/01/15/overview_001.dm3</nx:location>
      <nx:format>application/gatan-dm3</nx:format>
      <nx:description>Overview STEM image showing particle distribution</nx:description>
      <nx:preview>2024/01/15/overview_001.dm3.thumb.png</nx:preview>

      <!-- Metadata -->
      <nx:meta name="Data Type">STEM_Imaging</nx:meta>
      <nx:meta name="Creation Time">2024-01-15T10:30:45-05:00</nx:meta>
      <nx:meta name="Acceleration Voltage" unit="kV">200.0</nx:meta>
      <nx:meta name="Beam Current" unit="pA">500</nx:meta>
      <nx:meta name="Magnification">50000</nx:meta>
      <nx:meta name="Working Distance" unit="mm">5.2</nx:meta>
      <nx:meta name="Pixel Dwell Time" unit="us">10.0</nx:meta>
      <nx:meta name="Stage X" unit="mm">1.25</nx:meta>
      <nx:meta name="Stage Y" unit="mm">-0.75</nx:meta>
      <nx:meta name="Stage Z" unit="mm">0.0</nx:meta>
      <nx:meta name="camera_name">BM-Ceta</nx:meta>
    </nx:dataset>

    <!-- Additional datasets in this activity... -->

  </nx:acquisitionActivity>

  <!-- Additional acquisition activities... -->

</nx:Experiment>
```

**Key Elements:**

- **`<nx:Experiment>`**: Root element containing entire record
- **`<nx:summary>`**: Session-level metadata (who, what, why)
- **`<nx:instrument>`**: Instrument identification and location
- **`<nx:AcquisitionActivity>`**: Temporal grouping of related files
  - **`<nx:dataset>`**: Individual file metadata
    - **`<nx:acquisition>`**: Technical parameters with units and EM Glossary attributes
    - **`<nx:extensions>`**: Vendor-specific metadata not in standard schema
    - **`<nx:preview>`**: Thumbnail image location

## Navigating the XSD Documentation

The auto-generated XSD documentation below provides comprehensive details on all schema elements and types. Here's how to navigate it effectively:

### Schema Structure

The Nexus Experiment Schema is organized hierarchically:

1. **Root Element**: `Experiment` - Top-level container for all record data
2. **Session Metadata**: `summary` complex type - Who, when, what, why
3. **Instrument Information**: `instrument` complex type - Microscope details
4. **Activities**: `AcquisitionActivity` complex type - Temporal file groupings
5. **Datasets**: `dataset` complex type - Individual file metadata
6. **Acquisition Parameters**: `acquisition` complex type - Technical metadata with units

### Finding Specific Information

**To find required vs. optional elements:**
- Look for `minOccurs="0"` (optional) or `minOccurs="1"` (required)
- Elements without `minOccurs` specified default to `minOccurs="1"` (required)

**To understand element types:**
- Simple types (string, integer, etc.) are defined inline
- Complex types (nested elements) reference type definitions (e.g., `type="datasetType"`)
- Click type references to see full structure

**To see allowed values:**
- Enumerated types use `<xs:restriction>` with `<xs:enumeration>` values
- Example: `DatasetType` allows only: Image, Spectrum, SpectrumImage, Diffraction, Misc, Unknown

**To understand attributes:**
- Many quantity elements include a `unit` attribute for physical units
- EM Glossary elements include optional `emg:display_name` and `emg:id` attributes

### Common Elements Reference

| **Element Path** | **Purpose** | **Required** |
|------------------|-------------|--------------|
| `Experiment/summary/title` | Session title | Yes |
| `Experiment/summary/experimenter` | User information | Yes |
| `Experiment/instrument` | Microscope used | Yes |
| `Experiment/AcquisitionActivity` | File grouping | Yes (1+) |
| `AcquisitionActivity/dataset` | Individual file | Yes (1+) |
| `dataset/acquisition` | Technical metadata | No |
| `dataset/acquisition/extensions` | Vendor-specific data | No |

### Physical Quantities with Units

Metadata elements representing physical quantities use the `meta` element with a `unit` attribute:

```xml
<nx:meta name="Acceleration Voltage" unit="kV">200.0</nx:meta>
<nx:meta name="Working Distance" unit="mm">5.2</nx:meta>
<nx:meta name="Pixel Dwell Time" unit="us">10.0</nx:meta>
```

- **`name`** attribute: Human-readable parameter name (often from EM Glossary)
- **`unit`** attribute: Unit symbol (e.g., `"kV"`, `"nm"`, `"s"`)
- **Element value**: Numeric value in the specified unit

For dimensionless quantities, the `unit` attribute is omitted:

```xml
<nx:meta name="Magnification">50000</nx:meta>
```

Common units:
- Voltages: `kV` (kilovolts)
- Distances: `nm` (nanometers), `mm` (millimeters), `um` (micrometers)
- Times: `s` (seconds), `ms` (milliseconds), `us` (microseconds)
- Energies: `eV` (electron volts), `keV` (kiloelectron volts)
- Angles: `deg` (degrees), `rad` (radians), `mrad` (milliradians)

See {doc}`nexuslims_internal_schema` for the complete list of preferred units and unit handling details.

## See Also

- {doc}`nexuslims_internal_schema` - Internal backend metadata schemas and validation
- {doc}`em_glossary_reference` - EM Glossary field mappings
- {doc}`../user_guide/record_building` - How records are generated
- {doc}`../user_guide/taxonomy` - Data type classification