XRF - X-Ray Fluorescence

X-Ray Fluorescence (XRF) spectrometry is used to measure the elemental oxide composition of solid samples. It provides bulk oxide analysis expressed as weight percentages.

Method Overview

Property	Value
Full Name	X-Ray Fluorescence Spectrometry
Purpose	Measures elemental oxide composition of solid samples
Output Units	Weight percent (%)
SharePoint Location	Analytical Data > XRF > XRF Data
File Format	CSV

Oxides Measured

Oxide	Element	Description
CaO	Calcium	Calcium oxide
MgO	Magnesium	Magnesium oxide
Al2O3	Aluminum	Aluminum oxide
Fe2O3	Iron	Iron(III) oxide
SiO2	Silicon	Silicon dioxide
Na2O	Sodium	Sodium oxide
K2O	Potassium	Potassium oxide
TiO2	Titanium	Titanium dioxide
Cr2O3	Chromium	Chromium(III) oxide
MnO	Manganese	Manganese(II) oxide
SO3	Sulfur	Sulfur trioxide
SrO	Strontium	Strontium oxide
P2O5	Phosphorus	Phosphorus pentoxide
Cl	Chlorine	Chlorine
ZrO2	Zirconium	Zirconium dioxide
ZnO	Zinc	Zinc oxide
NiO	Nickel	Nickel(II) oxide
I	Iodine	Iodine
LOI	-	Loss on Ignition

Instruments

Batch files identify their instrument by a batch method name in the first column (e.g. SGA Purity V1, Rock_Primus_IV). Each batch method name is unique to one instrument, so the pipeline maps batch method name → instrument via BATCH_METHOD_MAP. This mapping is only for instrument attribution — it is unrelated to sample-prep classification (Pressed Pellet vs. Fused Bead vs. Filter), which is normalized separately (see below). A single instrument can run samples prepared multiple ways.

Primus IV (Primary)

The Primus IV is the primary XRF instrument at Brimstone and is prioritized in the data pipeline when duplicate measurements exist. It runs both pressed pellet and fused bead preparations.

Batch Method Names:

• SGA Purity V1
• SGA Purity V2
• Rock_Primus_IV
• Pressed Pellet
• CEM_28MAR2025
• CEM_28MAR2025_FP
• CEM_QuantPlusCl
• Alumina_bead

Supermini 200 (Secondary)

The Supermini 200 is the secondary instrument.

Batch Method Names:

• Rock
• XRF_CEM_011625

Sample Prep Methods

Type	Description
Pressed Pellet	Standard solid samples prepared as pressed pellets
Fused Bead	Samples prepared as a fused glass bead
Filter	Dissolved sample analysis (liquid XRF)

Method normalization: the method column is normalized to Fused Bead, Pressed Pellet, Filter, or Unknown (via normalize_xrf_method, independent of instrument/batch method name). The original raw value is preserved in method_raw. Methods that don't match a known pattern are labeled Unknown, not defaulted to Fused Bead.

Data Pipeline

SharePoint Source

• Site: Analytical Data
• Folder: XRF/XRF Data/
• File Type: CSV

Dagster Assets

The XRF data pipeline consists of the following assets in apps/datasmart/src/datasmart/assets/analytical/xrf.py:

xrf_incremental (sharepoint_multi_asset)
├── xrf_incremental_samples (backend.xrf_incremental_samples)
├── xrf_incremental_data (backend.xrf_incremental_data)
└── xrf_incremental_liquid (backend.xrf_incremental_liquid)
          ↓
    xrf (analytical.xrf)
    xrf_liquid (analytical.xrf_liquid)
          ↓
    xrf_simplified (analytical.xrf_simplified)
    xrf_simplified_duplicate_ids (analytical.xrf_simplified_duplicate_ids)

xrf_sgs (analytical.xrf_sgs) ← brimstone_us_geochemistry

Asset Descriptions

Asset	Schema	Description
xrf_incremental_samples	backend	Sample metadata from raw files
xrf_incremental_data	backend	Melted oxide values (long format)
xrf_incremental_liquid	backend	Liquid XRF oxide values
xrf	analytical	Wide format, 1 row per sample with oxide columns
xrf_liquid	analytical	Wide format for liquid XRF
xrf_simplified	analytical	Averaged replicates, Primus IV prioritized
xrf_simplified_duplicate_ids	analytical	Flagged duplicates for review
xrf_sgs	analytical	SGS external lab data

Database Tables

analytical.xrf

Wide format table with one row per sample.

Column	Type	Description
sample_id	string	Normalized sample ID
time_created	datetime	File creation time
CaO	float	Calcium oxide %
MgO	float	Magnesium oxide %
Al2O3	float	Aluminum oxide %
Fe2O3	float	Iron oxide %
SiO2	float	Silicon dioxide %
Na2O	float	Sodium oxide %
K2O	float	Potassium oxide %
TiO2	float	Titanium dioxide %
Cr2O3	float	Chromium oxide %
MnO	float	Manganese oxide %
SO3	float	Sulfur trioxide %
SrO	float	Strontium oxide %
P2O5	float	Phosphorus pentoxide %
Cl	float	Chlorine %
ZrO2	float	Zirconium dioxide %
ZnO	float	Zinc oxide %
NiO	float	Nickel oxide %
I	float	Iodine %
LOI	float	Loss on ignition %
instrument	string	Primus IV or Supermini 200
method_raw	string	Raw method value from the file (batch method name, "Unknown", or "liquid")
method	string	Sample preparation method
original_sample_id	string	Raw sample ID before normalization
file_loc	string	SharePoint file path
file_id	string	SharePoint file identifier

analytical.xrf_simplified

Cleaned version with:

• Averaged replicates
• Primus IV measurements prioritized
• LOI = 1.0 values filtered (likely placeholders)
• Metadata columns removed (except normalized method)
• Multiple rows per sample_id are possible when different methods exist

File Format Detection

The pipeline automatically detects file format:

Batch Files

• Method name in first column
• Multiple samples per file
• Examples: SGA Purity V1, Pressed Pellet, Rock

Single-Sample Files

• Metadata key-value pairs in first rows
• Sample type field present
• One sample per file

Liquid/Filter Files

• Sample type = Filter in metadata
• Uses Liquid conc. column for values

Duplicate Handling

Primus IV Prioritization

When duplicate sample IDs exist, the pipeline applies these rules:

1. Keep Primus IV: If any measurement is from Primus IV, keep only those
2. Keep all if no Primus IV: If no Primus IV measurements, keep all
3. Flag for review: All remaining duplicates go to xrf_simplified_duplicate_ids

LOI Filtering

Rows where LOI = 1.0 exactly are filtered out, as this is typically a default/placeholder value.

Redo File Detection

Files with these patterns in the filename are treated as redo files and prioritized:

• -Redo
• -redo
• _redo
• redo

Usage Examples

Query Simplified Data

from shared.db.sql import SQL

# Get all simplified XRF data
xrf = SQL.read("SELECT * FROM analytical.xrf_simplified")

# Filter by project team
xrf = SQL.read("""
    SELECT * FROM analytical.xrf_simplified
    WHERE sample_id LIKE 'P800%'
""")

Join with Other Analytical Data

combined = SQL.read("""
    SELECT
        x.sample_id,
        x.CaO, x.MgO, x.SiO2, x.LOI,
        i.Ca, i.Mg, i.Si
    FROM analytical.xrf_simplified x
    LEFT JOIN analytical.icp_simplified i
        ON x.sample_id = i.sample_id
""")

Common Calculations

import pandas as pd
from shared.db.sql import SQL

xrf = SQL.read("SELECT * FROM analytical.xrf_simplified")

# Sum of oxides
oxide_cols = ['CaO', 'MgO', 'Al2O3', 'Fe2O3', 'SiO2', 'Na2O', 'K2O', 'TiO2', 'MnO', 'P2O5', 'LOI']
xrf['oxide_sum'] = xrf[oxide_cols].sum(axis=1)

# Ca/Mg ratio
xrf['ca_mg_ratio'] = xrf['CaO'] / xrf['MgO']

# Silica ratio
xrf['silica_ratio'] = xrf['SiO2'] / (xrf['CaO'] + xrf['MgO'])

Related Assets

Asset	Description
sample_id_crosswalk	Maps messy sample IDs to canonical format
leach_crosswalk	Special mappings for leaching experiment IDs
project_team_map	Maps old process area codes to new project team codes
brimstone_us_geochemistry	SGS external lab data source

Troubleshooting

Duplicate Sample IDs

Check the xrf_simplified_duplicate_ids table for samples that need manual review:

duplicates = SQL.read("SELECT * FROM analytical.xrf_simplified_duplicate_ids")

Missing Samples

If a sample is missing from xrf_simplified:

1. Check if it exists in analytical.xrf (before simplification)
2. Check if sample ID normalization failed (look for NULL sample_id)
3. Verify the source file exists in SharePoint

Instrument Verification

To verify which instrument was used for a sample:

SQL.read("""
    SELECT sample_id, instrument, method
    FROM analytical.xrf
    WHERE sample_id = 'P800-001-1-QBND-FM'
""")