README - Rebasing and Splicing Notes

Project:
C:\Users\tokay\Dropbox\ReisMacroLab\PostMortem\2-PC\Final_Replication - Revised

Date:
2026-02-16

====================================================================
1) Goal
====================================================================

This note documents only the rebasing/splicing work used in the vintage pipeline:

- convert index levels to a consistent 2017=100 scale,
- handle the base/definition switch around 2023Q3,
- clarify which files are generated and which ones are actually used downstream.


====================================================================
2) Two rebasing workflows used in this project
====================================================================

A) Generic rebase helper:
- Notebook: Data_vintages.ipynb
- Purpose: read a raw CSV index series, optionally link at a known break, then rebase to 2017=100.

B) First-release + splice workflow (the _v2 generator):
- Notebook: Data_vintages_properVF.ipynb
- Purpose: build first-release real-time series and splice old-base to new-base regime before quarterly output.


====================================================================
3) Data_vintages.ipynb (generic rebase)
====================================================================

Main logic:
1. Read input CSV (date + value).
2. Optional break-link at BREAK_YEAR/BREAK_QUARTER (currently 2023Q3):
   k_link = value(2023Q2) / value(2023Q3)
   For t >= 2023Q3: value_linked(t) = value_raw(t) * k_link
3. Rebase linked series to 2017 average = 100:
   base_2017 = mean(value_linked in 2017)
   scale = 100 / base_2017
   value_2017eq100(t) = value_linked(t) * scale
4. Save output CSV with suffix:
   _rebased_2017eq100.csv

Files typically produced:
- Data/PCECTPI_20200429_rebased_2017eq100.csv
- Data/PCE_food_2020Q1vint_rebased_2017eq100.csv

Important:
- In current Create_alldata_* code, these rebased CSVs are not directly mapped.
- They are mainly helper/diagnostic products.


====================================================================
4) Data_vintages_properVF.ipynb (_v2 creation)
====================================================================

This is the key notebook for creating _v2 initial-release files.

4.1 PCE aggregate (PCEPI path)
- Input:  Data/PCEPI_1.xlsx (sheet: "Obs. By Real-Time Period")
- Output: Data/Initial_Release_PCE_v2.xlsx
- Value column: PCEPI
- Switch date: SWITCH_RT_START = 2023-09-29

4.2 PCE including energy+food
- Input:  Data/DNRGRG3Q086SBEA_1.xlsx (sheet: "Obs. By Real-Time Period")
- Output: Data/Initial_Release_PCE_IncludingEnergyandFood_v2.xlsx
- Value column: DNRGRG3Q086SBEA
- Switch date: SWITCH_RT_START = 2023-09-28

Core method:
1. For each period_start_date, keep FIRST release (minimum realtime_start_date).
2. Identify overlap periods around the regime switch where both:
   - last old-base vintage exists (realtime_end_date = switch - 1 day),
   - first new-base vintage exists (realtime_start_date = switch).
3. Compute splice factor as average ratio:
   splice_factor = mean(new_base_value / old_base_value over overlap periods)
4. Convert pre-switch first-release values:
   value_2017base = value_raw * splice_factor (if realtime_start_date < switch)
   value_2017base = value_raw               (otherwise)
5. Aggregate to quarter (mean or last, notebook parameter) and keep 2020Q2+.
6. Export single-sheet _v2 Excel.

Meaning:
- _v2 files are the cleaned, spliced, first-release 2017-base sources used by current Create_alldata_* notebooks.


====================================================================
5) Where rebased/spliced outputs are used downstream
====================================================================

Current Create_alldata mappings use:
- Data/Initial_Release_PCE_v2.xlsx -> pceall
- Data/Initial_Release_PCE_IncludingEnergyandFood_v2.xlsx -> pceEnergy

So the _v2 outputs from Data_vintages_properVF.ipynb are operational inputs.


====================================================================
6) Quick validation checks
====================================================================

After regenerating rebased/spliced files:

1. Check no artificial step change from base switch at 2023Q3.
2. Confirm scale consistency around 2017=100 (for series where that normalization is intended).
3. Confirm _v2 files contain 2020Q2+ quarterly first-release values.
4. Confirm Create_alldata_newvintages.ipynb and Create_alldata_oldvintages.ipynb still point to _v2 files.

End.
