IFs Base comparison
This IFs Base Comparison process relates to the consolidation of big data update. It is usually conducted once a year during the big data update.
Top down approach implies using all new updated data to build a new base and comparing with old/existing base. After few observations, we can replace few new series by old series to see the impact of addition of those new series. This page follows this approach.
It is the opposite of top-down approach. This allows us to add one new series at a time and observe its impact.
Things to do first before IFs Base Comparison:
- Rebuilding Base
- Variable names
- Variable names to compare model are in page Variable names
IFs Base Comparison
- Things to do for IFs base comparison
- See Data_listed as an example.
- In that excel file you can see tabs for variables and the world.
- For all variable world data is generated for 2015 and 2100. The measure used is ratio measure to compare old and new base. This allows us to see which variables deviate most in 2015 and 2100 relative to old base.
- The second tab all “All Variables_countries” is cumulation of all other individual variable tabs.
- Then you can see all important variables. Each variable has all countries and 2015 & 2100 raw data. Then we can calculate different ratio from here.
- Check all series and their differences between old and new data. This is very important to see impacts during initialization data.
- Start with important variables such as: GDP, POP, EDYRSAG15. See long term model impacts in these variables.
- It is also very important to see differences in initialization year. This might explain the long-term impact. In the end, we are trying to find what changes in data causing such model behavior.
- If data are vastly different during initialization year, check the data source and find the cause of those data difference.
- Check drivers. Drivers for a variable can be accessed in IFs. We have built a driver list for MFP and some other variables. Future RAs are expected to build more on this as they work. See “Drivers” folder to see more examples.
- See example of base comparison analysis.
- See “Base_comparison_analysis_write_up_examples_08142020.docx”
- See “Model Vetting Note Links_Examples_01012020.docx”
- Some other tips for base comparison.
- On variables that go near 0 or change signs, such as MFP, ratios are not really useful, but differences are. So, check the raw values before doing ratio comparison.
- As you work on this make a list of other tips that you think is beneficial for future RAs. You can add those here once you are done with this work.
Note: This guide is supposed to be a help book for RAs. By any means, it is not a comprehensive list of things to do as different base comparisons have their own uniqueness and should be consulted with your supervisor about what direction should you embrace.