Introduction to Pandas for Physics: Descriptive Statistics & Uncertainty Analysis - Computational Physics Formulation & Simulation

Program 1

Descriptive Statistics & Uncertainty Analysis

Python 3.11 Ready

import pandas as pd
import numpy as np

# High-precision student laboratory measurements of Simple Pendulum period T (seconds)
# For length L = 1.000 m. Note: 2.450 s represents an accidental stopwatch blunder
raw_trials = [
    2.008, 2.012, 1.995, 2.004, 2.019,
    2.001, 1.998, 2.450, 2.007, 2.011,
    2.005, 1.999, 2.015, 2.003, 2.006
]

df_pendulum = pd.DataFrame({
    'trial_id': range(1, len(raw_trials) + 1),
    'period_s': raw_trials
})

# 1. Comprehensive Summary Statistics using .describe()
print("=== Initial Descriptive Statistics (with Outlier) ===")
print(df_pendulum['period_s'].describe().round(4))

# 2. Outlier Detection via Tukey's Interquartile Range (IQR) Method
Q1 = df_pendulum['period_s'].quantile(0.25)
Q3 = df_pendulum['period_s'].quantile(0.75)
IQR = Q3 - Q1
lower_bound = Q1 - 1.5 * IQR
upper_bound = Q3 + 1.5 * IQR

print(f"\nIQR: {IQR:.4f} s | Acceptance Window: [{lower_bound:.4f} s, {upper_bound:.4f} s]")

# Filter out experimental blunders
is_valid = (df_pendulum['period_s'] >= lower_bound) & (df_pendulum['period_s'] <= upper_bound)
df_clean = df_pendulum[is_valid].copy()
blunders = df_pendulum[~is_valid]

print(f"Detected & removed {len(blunders)} blunder(s): {blunders['period_s'].tolist()} s")

# 3. Precision Statistical Reduction on Cleaned Trials
N = len(df_clean)
mean_T = df_clean['period_s'].mean()
median_T = df_clean['period_s'].median()
std_T = df_clean['period_s'].std(ddof=1)  # Sample standard deviation (N-1)
sem_T = df_clean['period_s'].sem()        # Standard Error of the Mean: s / sqrt(N)

print("\n=== Refined Experimental Statistics ===")
print(f"Number of Valid Trials (N):    {N}")
print(f"Mean Period (<T>):             {mean_T:.4f} s")
print(f"Median Period:                 {median_T:.4f} s")
print(f"Sample Standard Deviation (s): {std_T:.4f} s")
print(f"Standard Error of Mean (SEM):  {sem_T:.4f} s")

# 4. Error Propagation to Gravitational Acceleration: g = 4 * pi^2 * L / T^2
L = 1.000  # meters
g_exp = (4.0 * (np.pi ** 2) * L) / (mean_T ** 2)
delta_g = g_exp * 2.0 * (sem_T / mean_T)

print(f"\nExperimental Acceleration due to Gravity:")
print(f"  g = ({g_exp:.3f} +/- {delta_g:.3f}) m/s^2")
print(f"  Accepted Reference: g = 9.807 m/s^2")

Click "Run Code" to execute script and render figures.

Mathematical Problem Formulation

Statistical Metrics in Experimental Physics:

Given $N$ independent measurements $x_1, x_2, \dots, x_N$ of a physical observable:

1. Sample Mean ($\bar{x}$):

$$\bar{x} = \frac{1}{N} \sum_{i=1}^N x_i$$

2. Sample Standard Deviation ($s_x$ with Bessel's Correction):

$$s_x = \sqrt{\frac{1}{N - 1} \sum_{i=1}^N (x_i - \bar{x})^2}$$

3. Standard Error of the Mean ($\mathrm{SEM}$):

$$\mathrm{SEM} = \frac{s_x}{\sqrt{N}}$$

4. Outlier Rejection via Tukey's Interquartile Range (IQR):

Let $Q_1$ and $Q_3$ represent the 25th and 75th percentiles. A measurement is classified as a gross experimental error (blunder) if it falls outside the Tukey fences:

$$x_{\text{outlier}} \notin [Q_1 - 1.5 \cdot \mathrm{IQR},\; Q_3 + 1.5 \cdot \mathrm{IQR}]$$

5. Error Propagation to Derived Quantities:

For a simple pendulum of length $L$ and period $T$, $g = \frac{4\pi^2 L}{T^2}$. The propagated uncertainty is:

$$\delta g = g \cdot 2 \frac{\mathrm{SEM}_T}{\bar{T}}$$

Theoretical Background & Computational Mechanics

1. Precision vs Uncertainty in Experimental Physics:

In scientific measurement, stating a result as a single number without an uncertainty interval is meaningless. Pandas provides specialized functions for statistical reduction:

.mean(): Best estimate of the true physical parameter.
.std(ddof=1): Sample standard deviation reflecting experimental spread (uses $N-1$ degrees of freedom by default).
.sem(): Standard error of the mean, quantifying the uncertainty in the mean estimate.

2. Automated Outlier Rejection:

Student laboratory sessions often suffer from blunders (e.g. premature stopwatch clicks or electrical glitches). Tukey's IQR rule provides a principled, objective method to prune outliers without subjective investigator bias.

3. Error Propagation:

Combining the calculated $\mathrm{SEM}$ with analytical error propagation yields the final experimental report formatted according to international metrology standards: $g = (\bar{g} \pm \delta g)\,\text{m/s}^2$.

Theoretical Principles & Computational Physics Guide

Descriptive Statistics & Uncertainty Analysis: Computational Physics Modeling & Analysis

Dr. Alorika Chatterjee & Dr. Anirban Shaw Science & Physics Learners Python 3 / NumPy / SciPy

1. Core Principles & Theoretical Framework

Core Concept: Computational formulation and simulation of Descriptive Statistics & Uncertainty Analysis within the study of Introduction to Pandas for Physics.

Algorithmic Translation: Translates theoretical governing equations into clean, discrete Python procedures and verified numerical solvers.

Numerical Verification: Evaluates numerical accuracy, parameter sensitivity, and error bounds for reliable computational results.

2. Mathematical Formulation & Algorithmic Pipeline

Governing computational formulation for Descriptive Statistics & Uncertainty Analysis:

$$ \mathcal{F}_{\text{model}}(\mathbf{x}, t; \mathbf{p}) = 0 \implies \mathbf{y}_{n+1} = \mathcal{S}(\mathbf{y}_n, \Delta t, \mathbf{p}) $$

1. Define Physical Parameters & Boundaries → 2. Execute Numerical Simulation Steps → 3. Evaluate Observables & Plot Results

3. Numerical Mechanics & Implementation Logic

Parameter Initialization: Set physical constants, initial states, and coordinate boundary bounds for Descriptive Statistics & Uncertainty Analysis.
Step Execution: Advance the computational state using optimized vector operations and scientific libraries.
Convergence Control: Monitor stability and ensure simulation metrics remain physically meaningful.

4. Relevant Computational Practice Exercises

Exercise 1

Implement Descriptive Statistics & Uncertainty Analysis and observe how the output responds to variations in primary physical parameters.

Exercise 2

Refine the calculation grid or step size to observe convergence and reduction in numerical approximation error.

Exercise 3

Compare the simulated computational results against an analytical benchmark or limiting physical case.

Standard Literature & References:

Mark Newman, "Computational Physics", University of Michigan (2013).
Hans Petter Langtangen, "A Primer on Scientific Programming with Python", Springer (2016).

Ch. 8.6 Descriptive Statistics & Uncertainty Analysis