From 6c055c8c23a4c72195bdc210e9e3069c8f64387e Mon Sep 17 00:00:00 2001 From: Oliver Jowett Date: Tue, 24 Jul 2018 17:31:43 +0100 Subject: [PATCH] Update aircraft db docs --- tools/README.aircraft-db | 40 ------------------------- tools/README.aircraft-db.md | 58 +++++++++++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+), 40 deletions(-) delete mode 100644 tools/README.aircraft-db create mode 100644 tools/README.aircraft-db.md diff --git a/tools/README.aircraft-db b/tools/README.aircraft-db deleted file mode 100644 index 3b53c9a..0000000 --- a/tools/README.aircraft-db +++ /dev/null @@ -1,40 +0,0 @@ -The dump1090 webmap uses a static database of json files to provide aircraft -information. - -This directory has some tools to turn a CSV file with aircraft data into -the json format that the dump1090 map expects. - -The default data comes from a database kindly provided by VRS (unfortunately -no longer updated). This data is in vrs.csv. It was extracted by: - - $ wget http://www.virtualradarserver.co.uk/Files/BasicAircraftLookup.sqb.gz - $ gunzip BasicAircraftLookup.sqb.gz - $ tools/vrs-to-csv.py BasicAircraftLookup.sqb >tools/vrs.csv - -You can modify vrs.csv (or build a new CSV entirely) and update the database. - -First, as an optional step, you can prune out all registrations which match -what the in-browser hexid-to-registration logic would generate anyway. This -requires nodejs, see the comments in filter-regs.js - - $ nodejs tools/filter-regs.js tools/vrs-pruned.csv - -Next, turn the pruned CSV into a set of json files: - - $ tools/csv-to-json.py tools/vrs-pruned.csv public_html/db - -The contents of public_html/db should be installed where the webmap can find -them; the Debian packaging puts these in -/usr/share/dump1090-mutability/html/db - -The CSV format is very simple. The first line must be a header line that names -the columns. These columns are understood: - - icao24: the 6-digit hex address of the aircraft - r: the registration / tail number of the aircraft - t: the ICAO aircraft type of the aircraft, e.g. B773 - -Any other columns are put into the json DB under the name you give them, but -the standard map code won't do anything special with them. You can pick these -columns up in the PlaneObject constructor (see planeObject.js where it calls -getAircraftData()) for later use. diff --git a/tools/README.aircraft-db.md b/tools/README.aircraft-db.md new file mode 100644 index 0000000..8c1638c --- /dev/null +++ b/tools/README.aircraft-db.md @@ -0,0 +1,58 @@ +# dump1090 aircraft database + +The dump1090 webmap uses a static database of json files to provide aircraft +information. + +This directory has some tools to turn a CSV file with aircraft data into +the json format that the dump1090 map expects. + +## Data sources + +The default data comes from a combination of: + + * some historical data kindly provided by VRS (unfortunately no longer + updated) - this data is in vrs.csv.xz; and + * a data export provided by FlightAware that is updated periodically - + this data is in flightaware-*.csv.xz + +The VRS data was extracted by: + +```sh +$ wget http://www.virtualradarserver.co.uk/Files/BasicAircraftLookup.sqb.gz +$ gunzip BasicAircraftLookup.sqb.gz +$ tools/vrs-to-csv.py BasicAircraftLookup.sqb >tools/vrs.csv +``` + +The FlightAware data is a subset of the registry information that FlightAware +uses internally. It contains only the data that FlightAware can redistribute +to the public; some data sources that FlightAware uses do not allow this and +are excluded from the export. + +## Regenerating the json database + +To regenerate the json database from these input files: + +```sh +$ rm ../../public_html/*.json +$ xzcat vrs.csv.xz | nodejs ./filter-regs.js >vrs-pruned.csv +$ xzcat flightaware-20180720.csv.xz | nodejs ./filter-regs.js >fa-pruned.csv +$ ./csv-to-json.py vrs-filtered.csv fa-filtered.csv ../public_html/db +``` + +Additional CSV files can be given to `csv-to-json.py` if desired. + +The contents of public_html/db should be installed where the webmap can find +them; the Debian packaging puts these in +/usr/share/dump1090-mutability/html/db + +The CSV format is very simple. The first line must be a header line that names +the columns. These columns are understood: + +* icao24: the 6-digit hex address of the aircraft +* r: the registration / tail number of the aircraft +* t: the ICAO aircraft type of the aircraft, e.g. B773 + +Any other columns are put into the json DB under the name you give them, but +the standard map code won't do anything special with them. You can pick these +columns up in the PlaneObject constructor (see planeObject.js where it calls +getAircraftData()) for later use.