Wellwiki Page Generation Scripts

Page Generation

Wellwiki uses four primary scripts in order to generate new pages. All of these scripts are located within ~/installers/wikimanager/

pre_deployment.sql

The first script updates the violation counts for wells in the well table of wellsdb_dev. It can be run from mysql by logging in as root, selecting wellsdb_dev as the table of choice, and running source ~/installers/wikimanager/pre_deployment.sql within a mysql console.

The script takes about 1 minute to fully execute.

well_updates.py

This script updates the well table based on information in permit, production, spud, municipality, and county. It should be run after new data is added to the database to ensure that well info is fully populated. It is located in ~/scripts/ and is run using python well_updates.py in this directory. If only specific update functions need to be carried out, this can be specified in if __name__=='__main__':.

UpdateManagerPageTable.py

This is a python2 script which adds any wells within the mysql wellsdb_dev database to the wikiManagerPages table within the wikidb database. This should be run prior to CreateEntitySources such that all necessary pages needed to be generated are able to be flagged for updates.

CreateEntitySources.py

This is a python2 script that will read through all wiki pages located in the wikiManagerPages table within the wikidb database that have "NEEDS_UPDATE" set to 1.

It uses scripts within DBUtil/ to query the mysql database, which in turn return dictionaries for each well in question in which keys correspond with column names from the mysql database, and values are the corresponding entries for a given well. Scripts within FileUtil/SourceBuilders/ generate page templates and output parsed files using well specific data from DBUtil results. Note that the *SourceBuilder.py scripts are used, while the *TableTemplateBuilder.py scripts are older versions of these and are therefore unused.

Each entry within the wikiManagerPages table will be queried for in the wellsdb_dev database, which then outputs newly generated pages to ~/PycharmProjects/WikiManager/WikiSources

Here is some sample SQL to flag all Pennsylvania pages for update:

UPDATE `wikiManagerPages`

SET `NEEDS_UPDATE`='1'

WHERE `PAGE_NAME` LIKE '37%'

This script is currently fully functional for at least the state of Pennsylvania.

commit_staged_sources.py

This python2 file takes a source directory as an argument, and begins adding all wiki pages within that source directory to the actual wikimedia database, making them available for viewing here.

Usage: python commit_staged_sources.py ~/PycharmProjects/WikiManager/WikiSources