Volatility surface research with end-of-day options data

Which option contracts can be combined in a volatility surface?

A volatility surface needs comparable option payoffs and a consistent valuation basis. Group observations by date, underlying, contract root and expiration, and check style and settlement_time. SPX and SPXW share an underlying index, but their AM/PM settlement conventions must be handled before observations are combined.

A strike and underlying symbol do not establish an identical deliverable. HistoricalData.net's 34-column options CSV has no dedicated contract-multiplier or deliverable field. Investigate adjusted and non-standard contracts using contract specifications before combining them with ordinary series.

Choose a consistent call/put and moneyness policy. For example, a European smile can use out-of-the-money puts below the forward and calls above it, with a stated rule near the forward. Combining calls and puts at the same strike without a selection or weighting rule can count the same part of the smile twice.

What does iv_flag establish about an IV observation?

In HistoricalData.net options files, iv_flag identifies the midpoint IV calculation status and forward convention. The flag describes a model calculation, not whether the quote is fresh or liquid.

iv_flag = 0

Midpoint IV and all five Greeks are populated using a parity-derived forward.

iv_flag = 1

Midpoint IV and all five Greeks are populated using the fallback F = S × exp(rT), which ignores dividends.

iv_flag = 2–7

Midpoint IV and all five Greeks are blank. The flag distinguishes missing inputs, an unusable midpoint, solver failures and non-positive effective time to expiry.

Options iv_bid and iv_ask are solved independently of midpoint IV. Either side can be blank when midpoint IV is present, or populated when midpoint IV is blank. Missing IV means no published solution under the calculation rules; it does not mean zero volatility.

Compare a fitted surface with and without flag-1 rows, and report exclusions by expiration. The fallback forward changes the dividend assumption but does not automatically create a discontinuity. See all IV status definitions and pricing inputs and approximations.

How should quotes and bid-ask IV ranges be filtered?

In HistoricalData.net options files, iv_ask - iv_bid measures the range of volatilities implied by the quoted bid and ask under the pricing model. The range is not a statistical confidence interval or a measured IV error. A tight range also does not establish quote freshness.

The options generator requires both bid > 0 and ask > 0 to form a midpoint. Missing quote sides must remain missing; do not replace them with zero or use half the ask as a midpoint. Check quote ordering and the availability of all IV values required by the filter.

A preliminary filter in pandas

Run this example with pandas installed, from the extracted options sample folder. The example selects SPXW European, PM-settled calls on 24 August 2022. The maximum IV width of 0.05 is an illustrative five volatility percentage points, not a universal quality threshold.

import pandas as pd

df = pd.read_csv("day_by_date/2022-08-24_options.csv")
chain = df.loc[
    df["underlying"].eq("SPXW")
    & df["quote_date"].eq("2022-08-24")
    & df["style"].eq("E")
    & df["settlement_time"].eq("PM")
    & df["type"].eq("call")
].copy()

solved = chain["iv_flag"].isin([0, 1])
quotes = chain["bid"].gt(0) & chain["ask"].ge(chain["bid"])
ivs = chain[["iv_bid", "iv", "iv_ask"]]
in_solver_range = ivs.apply(lambda s: s.between(0.0001, 5.0)).all(axis=1)
ordered = (
    chain["iv_bid"].le(chain["iv"])
    & chain["iv"].le(chain["iv_ask"])
)
width = chain["iv_ask"] - chain["iv_bid"]

keep = solved & quotes & in_solver_range & ordered & width.le(0.05)
usable = chain.loc[keep].copy()
print(f"Retained {len(usable)} of {len(chain)} SPXW call rows")
print(usable.groupby("expiration")["strike"].nunique())

The bounds 0.0001 to 5.0 are the archive's IV solver interval. The example checks published values and retains only rows with complete bid/mid/ask IV; it neither recalculates IV nor fits a surface. Each expiration still needs enough usable strikes, and a calls-only subset may give poor coverage of the low-strike wing.

Surface filters should record exclusions by expiration, moneyness and IV flag. Test different spread thresholds and call/put policies. Quote age cannot be tested with this 2022 sample because its quote_time values are blank.

Are end-of-day option quotes synchronized?

HistoricalData.net end-of-day options files contain each contract's last standing bid/ask quote. The bid and ask within a row belong to the same quote record, but different contracts can have different quote ages. A daily file does not establish a synchronized snapshot across strikes and expirations.

Options quote_time is available from 2026-08-06 where reported, in UTC, and can precede quote_date. Measure quote age against a defined valuation cutoff using the product's trading calendar and timezone, including daylight saving and shortened sessions. Earlier rows, including the 2022 sample, have no timestamp from which to calculate quote age.

Options underlying_close normally uses the unadjusted official close, with documented historical and index exceptions. The underlying price is not guaranteed to be contemporaneous with each option quote. Do not substitute a split-adjusted stock price without also reconciling the option's strike and deliverable.

How is time to expiry calculated in the options archive?

Contractual expiration, last trading time and settlement time are different quantities. HistoricalData.net options calculations use the calendar-day difference between expiration and quote_date, divided by 365, with the documented deductions for Saturday expirations and AM settlement.

The options model subtracts one day for the historical Saturday-expiration convention and one day for AM settlement; both deductions can apply. Effective time at or below zero produces iv_flag = 6 and blank IV and Greeks. PM expiry-day rows and AM rows on the preceding day can therefore retain quotes while the calculated values are blank.

The archive's time-to-expiry calculation is a daily model convention. A missing solution at its zero-time boundary does not establish that the contract had no remaining economic time value. To use an intraday valuation clock, recalculate IV with consistent prices, forward, rates and time inputs. See the expiry convention.

Which pricing models and inputs determine the published IV?

HistoricalData.net options IV is the volatility parameter obtained by solving a pricing model against a quoted price. The result depends on exercise style, time, forward and rate inputs; it is not a direct measurement or guarantee of future realized volatility.

European exercise

Black-76 on the forward, with analytic Greeks.

American exercise

A 101-step Leisen-Reimer binomial tree with early exercise and an implied continuous dividend yield. Greeks use finite differences on the same tree.

Forward selection

Within each contract-root and expiration group, select the strike nearest the underlying price with positive call and put quote sides. Compute F = K + exp(rT) × (call_mid - put_mid); otherwise use the documented fallback if no usable pair or positive forward exists.

Rate convention

US Treasury daily par yields are linearly interpolated to maturity, with flat extrapolation and the documented missing-curve rules. Using a bootstrapped discount curve would be a different rate convention.

Put-call parity is an exact pricing relationship for matching European payoffs under its assumptions. Applying the same formula to American quotes is a forward-estimation approximation because early-exercise premiums can affect the call/put difference. The archive's inferred continuous yield also differs from a schedule of discrete dividends. A flag-0 result does not remove these modelling limits.

The options CSV does not contain separate forward, rate or implied-dividend-yield columns. Reproducing published IV requires the relevant call/put chain, rate inputs and pair-selection and model conventions, even when the final fit uses only one option type.

Options IV is annualized and expressed as a decimal: 0.20 means 20%. Published theta is per year, while vega and rho are per 1.00 change in volatility and rate. Divide theta by 365 for the stated daily convention, and vega or rho by 100 for one percentage point. See all field units.

How do you check a fitted volatility surface?

A visually smooth implied volatility surface is not sufficient evidence of a consistent fit. Reprice retained contracts under the same model and compare price residuals with their bid–ask spreads. An IV difference can have a very different price impact at different vegas.

Strike consistency

For comparable European calls at one expiry, fitted prices should be non-increasing and convex in strike, and satisfy the relevant price bounds. Check between observed strikes as well as at them.

Maturity consistency

In the European SVI setting with proportional dividends, total implied variance w(k,T) = iv(k,T)^2 × T must be non-decreasing in T at fixed forward log-moneyness k = ln(K/F(T)). IV itself need not increase with maturity. Use the same forward and discount conventions throughout.

Validation and wings

Hold out strikes to assess interpolation, and use later dates to evaluate the stability of the fitting procedure. Report errors, exclusions and strike coverage by expiry; state the extrapolation rule for sparse wings.

In the surface formulas, K is strike, F(T) the forward for that expiry, T time in years and iv annualized decimal volatility. European no-arbitrage conditions cannot be transferred unchanged to raw American-option quotes. A finite-grid check can reveal a violation but does not prove the entire interpolated surface is arbitrage-free.

For the European SVI setting: Gatheral and Jacquier, Arbitrage-free SVI volatility surfaces.

How do market regimes and field availability affect comparisons?

The HistoricalData.net options archive begins on 8 February 2002, with coverage varying by underlying, contract and field. Compare separate historical windows and input populations; an aggregate fit score can hide changes in spreads, available expirations and missing values.

2008–09 financial crisis

Inspect quote spreads, missing solutions and fit stability during large market moves.

2020 COVID crash

Compare the rapid change in skew and term structure with calmer surrounding periods.

2022 bear market

Evaluate the fit in a sustained declining market; July–December files are available in the free sample.

Options quote sizes begin on 2005-01-03; open, high, low, trade_vwap, transactions and active_minutes begin on 2014-06-02. multileg_volume begins on 2019-11-04, and quote_time on 2026-08-06. Availability dates do not guarantee a value in every later row.

A filter requiring a field unavailable in an earlier era removes that era by construction. Keep missing-field exclusions separate from market-regime effects, and do not turn blank historical fields into zero activity. Consult the options field table before comparing periods.

What can end-of-day options data establish?

HistoricalData.net options files provide 34-column contract observations for daily smile, skew, term-structure and event studies. The archive provides neither an intraday quote sequence nor order-book depth, fitted surfaces or a complete contract-deliverable history. A recorded quote or midpoint does not establish an executable fill.

Options backtests need explicit signal timing, execution prices, costs and contract-settlement accounting. A signal constructed from final end-of-day observations cannot assume a fill at those same observations. A historical surface fit is a description of selected quotes under a model, not evidence of an executable strategy or a forecast of realized volatility.