Show HN: CIA World Factbook Archive (1990–2025), searchable and exportable

(cia-factbook-archive.fly.dev)

231 points | by MilkMp 9 hours ago ago

57 comments

  • 1659447091 an hour ago ago

    There is a github of the factbook for anyone that just wants JSON or markdown files:=> https://github.com/factbook

    "A cache for datasets for the country profiles from the World Factbook in the original (1:1) format from the cia.gov website"

    https://github.com/factbook/cache.factbook.json

    • MilkMp an hour ago ago

      Hi there, thanks for linking this! My GitHub and website both link to and use this source! I just thought putting it in a SQL database and making the entire 1990-2025 queryable was needed since I couldn't find one anywhere :)

  • freakynit 42 minutes ago ago

    To the author:

    In case you are patching fields/bugs in database (like country codes for example), would it be possible for you to share that database as well with us so we can build on top?

    This is actually an excellent dataset to test GraphRAG capabilities.

    Also, a world simulation game, embodied with real data and real changes, can be built based off this data.

    Thanks..

    • MilkMp 34 minutes ago ago

      Hey there, yeah, definitely. I maintain .txt change logs for all data modifications. To be clear, no information is added or altered — the Factbook content is exactly what the CIA published. The parsing process structures the raw text into fields (removing formatting artifacts, sectioning headers, and deduplicating noise lines), but the actual data values are untouched. What I've added on top are lookup tables that map the CIA's FIPS 10-4 codes to ISO Alpha-2/3 and a unified MasterCountryID, so the different code systems can be joined and queried together.

      I will add them to the github :)

  • freakynit an hour ago ago
  • roysting 3 hours ago ago

    Hi. Nice project. One issue though; if you go to the Factbook for any year[1], the link to the entry for “Germany”[2] will take you to the entry for the Gambia for every year I have checked. I have not noticed any other countries where that happens.

    [1] https://cia-factbook-archive.fly.dev/archive/2002

    [2] https://cia-factbook-archive.fly.dev/archive/2002/GM

    • tjsch 2 hours ago ago

      I found another example: searching for "Nicaragua" takes you to the page for "Niger".

      • MilkMp 2 hours ago ago

        Hi there, I have located the root cause and will be fixing the issue:

        Root cause: CIA uses FIPS codes (CanonicalCode), which differ from ISO Alpha-2 for many countries. Templates and SQL queries prioritized CanonicalCode over ISOAlpha2, so URL codes like /archive/2025/AU matched the wrong country.

        Australia (AU) -> American Samoa (AS = CIA FIPS for Australia) Singapore (SG) -> Senegal (SG = CIA FIPS for Senegal) Germany (DE) -> Gambia (GM = CIA FIPS for Germany)

  • b8 7 hours ago ago

    2025-2026 is available (to purchase/read outside or ur site) and the last version 2026-2027 is planed for release on April 7th, https://www.amazon.com/CIA-World-Factbook-2026-2027-ebook/dp....

    • crims0n 2 hours ago ago

      Somehow it escaped me that these were published books as well. Thank you kind stranger.

    • toomuchtodo 2 hours ago ago

      Internet Archive has 2025-2026 in their possession, should make it into OpenLibrary eventually once scanned.

    • MilkMp 7 hours ago ago

      Will add to this once it's out. Thanks for sending this :)

      • b8 2 hours ago ago

        I have the 2025-2026 one. Email me for it.

  • eddythompson80 2 hours ago ago

    Cool project. The world population seems to be double counted. I think https://cia-factbook-archive.fly.dev/analysis/trends

    • MilkMp 2 hours ago ago

      Found the root cause. The "World" entity (population ~8 billion) was being called alongside all individual countries, doubling the total. Thank you again!

    • MilkMp 2 hours ago ago

      Will fix right now! I think I was looking at this for too long and missed some things. Thank you :)

  • ggm 6 hours ago ago

    This is an archive of the service which is being shut down under the current WH administration?

    • 1f60c 5 hours ago ago

      Yes, that is correct.

  • kshri24 4 hours ago ago

    There is a bug in the time series charts. Data needs to be normalized prior to charting. For example: https://cia-factbook-archive.fly.dev/archive/field/IN/Broadb...

    • MilkMp an hour ago ago

      Found the problem, the total regex doesn't handle magnitude suffixes:

      2018: total: 17,856,024 → parses as 17856024 (correct raw count) 2020: total: 18.17 million → parses as 18.17 (WRONG - drops "million") 2025: total: 39.3 million → parses as 39.3 (WRONG) So the chart jumps from ~18 million down to ~18, making it wrong. The fix is to handle "million/billion/trillion" after total.

      Just deployed a new bug fix.

      Thanks for bringing this to my attention!

  • freakynit an hour ago ago

    Excellent site.

    One small bug though: https://cia-factbook-archive.fly.dev/analysis/compare?a=IN&b...

    .. The second dropdown switches to "Comoros" instead of "China" even after selection, though URL says CN for China.

    • MilkMp an hour ago ago

      Will check out! Thank you!

  • 3eb7988a1663 5 hours ago ago

    Just an incredible service. Really appreciate that you put all of your backend work into the open.

    • MilkMp an hour ago ago

      Thanks so much!

  • celeryd 7 hours ago ago

    Any way to download them all at once?

    • MilkMp 7 hours ago ago

      Hey there, will add the feature. Wasn't sure if people's computers could handle it all in one, lol, but will make it available in the data export page.

      • ngcc_hk 4 hours ago ago

        Not all. But some may. And in case you were shut down someone else can continue and one day it may be resurfaced perhaps even in USA.

  • cwnyth 5 hours ago ago

    Kudos! I was working on doing this as well, so it's nice to see it already done.

    • MilkMp an hour ago ago

      Hi there! If you have anything you want added to the site, just let me know :) I can definitely try.

  • ronald_petty 8 hours ago ago

    I like the timeline feature. Maybe I need to spend more time, but to see political changes / borders / etc. would all be great! Keep up the good work.

    • MilkMp an hour ago ago

      Ohh that is a great idea! And since we already have the political field in SQL!. I will start working on some of this and update the website this week. Thank you for the awesome suggestions!

  • FergusArgyll 7 hours ago ago

    Nice!

    One thing; you're supposed to write "Cannot confirm or deny my affiliation with the CIA"

    • sailfast 6 hours ago ago

      That’s a bit of a canary is it not? You don’t need to say that and wouldn’t know to say that unless you had worked in the space or wanted us to think you did :)

    • MilkMp 7 hours ago ago

      Thanks, I will change it!

  • nubg 6 hours ago ago

    Site loads very slowly for me. Tried various devices and networks. Same for a friend of mine overseas.

    • MilkMp 6 hours ago ago

      Will scale the website

      • nubg 5 hours ago ago

        To clarify, I am a shill for fly.io and wanted to get you to spend more money by scaling it up. The site loaded instantaneously on the first try, so fast I thought it was local.

      • Betelbuddy 6 hours ago ago

        You should include this one, will also go away soon most likely:

        https://www.cia.gov/resources/cia-maps/

        • MilkMp 6 hours ago ago

          I'll start working on this now! Thank you for sending it! It will be interesting to see if I can incorporate them into the globes or when the country info pops up!

  • daveelkan 5 hours ago ago

    found a bug: Australia links to American Samoa in 2025 archive.

    • stephen_g 4 hours ago ago

      Yes I noticed that too, and clicking on Austria takes you to Australia instead! (AU instead of AT which is Austria's country code)

      Then when you actually are in Australia, if you click back to 2001 or earlier it changes to 'Ashmore and Cartier Islands'

      • MilkMp 2 hours ago ago

        Hi there, I have located the root and sent out a bug fix.

        Root cause: The CIA World Factbook, published by the Central Intelligence Agency, uses the U.S. Government's FIPS 10-4 country codes, which differ from the ISO 3166-1 Alpha-2 codes used by the rest of the world. Of the 281 entities in our database, 173 have different FIPS and ISO codes. Our lookups matched FIPS codes first, so when codes collided between the two systems, the wrong country was loaded. Fixed all 13 queries and 6 templates to always prefer ISO over FIPS.

        Examples fixed:

        Australia (ISO=AU) was loading American Samoa (FIPS=AQ, but Australia's FIPS=AS collides with American Samoa's ISO=AS) Singapore (ISO=SG) was loading Senegal (FIPS=SG) Germany (ISO=DE) was loading Gambia (FIPS=GM = Germany's FIPS, ISO=GM = Gambia) Bahamas (ISO=BS) was loading Burkina Faso (FIPS=BF = Bahamas' FIPS, ISO=BF = Burkina Faso)

      • MilkMp 3 hours ago ago

        Will fix. Thank you! Most likely a grouping problem due to the MasterCountry ID.

    • gbennett71 3 hours ago ago

      Confirmed, affects multiple years' data

  • RobRivera 7 hours ago ago

    Hurray!

    I didnt discover this until I saw the recent post about its deactivation.

  • shevy-java 7 hours ago ago

    Hmm. It's kind of weird, because I think I actually used it in the 1990s, probably shortly before Wikipedia emerged. Ever since Wikipedia, I don't think I used the CIA world Factbook much at all, so in a way I guess this partly explains why the website is now defunct. But I am a tiny bit sad that it is gone, if only for a piece of nostalgia from the 1990s era. I think we need to be careful - yes, wikipedia has that information, but we kind of lose websites here. That is a potential danger, because we end up with more and more of a monopoly which is rarely good (ok, wikipedia may be an exception but it also has intrinsic quality issues; it is still excellent in many ways but not perfect, and we may get tunnel vision the more websites vanish - just look at the AI slop autogenerated "content" or "affiliate" links you see in a google search, if anyone is still using that).

    • MilkMp 7 hours ago ago

      Glad I was able to get the original fact book data that other archivists have gathered over the years- Project Gutenberg (plain text), Wayback Machine (HTML zips and factbook.jsons, and one from the agency's websites

    • cwnyth 5 hours ago ago

      Wikipedia has some consistency issues and often linked to the CIA Factbook as well.

    • orhmeh09 5 hours ago ago

      World facts provided by the CIA, too, have intrinsic quality issues. I'm not too worried!

  • ohyoutravel 6 hours ago ago

    This is pretty basic but kinda neat. A good way to browse the fact books like a website. Definitely could use more features but imo superior than flipping through a PDF.

    • MilkMp 6 hours ago ago

      Originally, my plan was just to create the archive, but I have expanded the scope, lol.

    • MilkMp 6 hours ago ago

      Hey, what features would you like to see??

      • ix101 6 hours ago ago

        Hi, thanks for this! Not sure if you're aware that clicking Australia goes to American Samoa, similar issue with some others that I encountered (Bahamas -> Burkina faso).

        • MilkMp 2 hours ago ago

          Hi there, I have located the root and sent out a bug fix.

          Root cause: The CIA World Factbook, published by the Central Intelligence Agency, uses the U.S. Government's FIPS 10-4 country codes, which differ from the ISO 3166-1 Alpha-2 codes used by the rest of the world. Of the 281 entities in our database, 173 have different FIPS and ISO codes. Our lookups matched FIPS codes first, so when codes collided between the two systems, the wrong country was loaded. Fixed all 13 queries and 6 templates to always prefer ISO over FIPS.

          Examples fixed:

          Australia (ISO=AU) was loading American Samoa (FIPS=AQ, but Australia's FIPS=AS collides with American Samoa's ISO=AS) Singapore (ISO=SG) was loading Senegal (FIPS=SG) Germany (ISO=DE) was loading Gambia (FIPS=GM = Germany's FIPS, ISO=GM = Gambia) Bahamas (ISO=BS) was loading Burkina Faso (FIPS=BF = Bahamas' FIPS, ISO=BF = Burkina Faso)

  • nephihaha 7 hours ago ago

    What is its copyright status?

    • MilkMp 7 hours ago ago

      The data from the CIA World Factbook is in the public domain (being a U.S. Government work) and is free for anyone to use. The ETL scripts and data tools available in the GitHub repository are open source and licensed under the MIT License. However, the web application itself is proprietary software, with all rights reserved.