If you find this page useful and would like to be notified of changes made to this page, start by inputting your email below.
powered by ChangeDetection
(updated for download information for compliance, spud dates, and permits) |
(→May 2015 Production Update: detail compliance download issue) |
||
| Line 25: | Line 25: | ||
CSVs for Compliance, Permits, and SPUD dates were obtained from http://www.portal.state.pa.us/portal/server.pt/community/oil_and_gas_reports/20297 using their respectively linked utilities. Maximum date range from 01/01/1800 to 05/26/2015 was used to obtain data. Default settings for all categories were appropriate except for "inspections with violations only" for compliance, which had to be set to "no" in order to properly obtain all compliance records. | CSVs for Compliance, Permits, and SPUD dates were obtained from http://www.portal.state.pa.us/portal/server.pt/community/oil_and_gas_reports/20297 using their respectively linked utilities. Maximum date range from 01/01/1800 to 05/26/2015 was used to obtain data. Default settings for all categories were appropriate except for "inspections with violations only" for compliance, which had to be set to "no" in order to properly obtain all compliance records. | ||
| + | |||
| + | Compliance does not seem to produce results properly. Even when selecting a date range for a singular day, the error "An error occurred while executing OnInit: Date Received From should not be later than Date Received To (rsErrorInOnInit)" occurs. Otherwise queries for the data merely time out. | ||
Updating Production data. Extra column for WELL_API had to be added, using Well Permit #. Excel formula ="37-"&B2 was used to produce WELL_API values. Well Permit # renamed to WELL_API_COUNTY_ID. Extra columns WELL_COUNTRY set to "United States" and WELL_STATE_CODE set to "PA".
Needed to specify that lines were delimited by \n instead of auto to get around PHP "invalid comma count error." There was a typo in one cell for a sheet that had a "\" character, which produced another invalid column count error. Also renamed all columns in CSV to columns in SQL db as follows:
WELL_API, WELL_API_COUNTY_ID, PERIOD_ID, PRODUCTION_INDICATOR, WELL_STATUS, FARM_NAME,WELL_ID, SPUD_DATE, GAS_QUANTITY_MCF, GAS_PRODUCTION_DAYS, CONDENSATE_QUANTITY_BBL, CONDENSATE_PRODUCTION_DAYS, OIL_QUANTITY_BBL, OIL_PRODUCTION_DAYS, AVERAGED, OPERATOR_NAME, OPERATOR_ID, WELL_COUNTY, WELL_MUNICIPALITY, LATITUDE_DECIMAL, LONGITUDE_DECIMAL, UNCONVENTIONAL, CONFIGURATION, HOME_USE, REPORTING_PERIOD, COMMENT_REASON, COMMENT_TEXT, WELL_COUNTRY, WELL_STATE_CODE
which needs to be entered on the import screen for PHPmyAdmin to upload correctly for column headers.
This has been more work than it should have been. There is a CSV module in python if we elect to go for full automation, so this could possibly be done automatically.
In addition to removing the \ character from CSVs, they must also be processed to remove the quotation character " to allow for proper importation. Older yearly CSV files (non-monthly) were split using a CSV split utility in order to meet MySQL's 2048KiB filesize limit
Duplicate entries were discovered in the production table. Entries were removed by producing a new temporary table with the same structure but a primary key consisting of the WELL_ID and production date. Data was copied over with a INSERT IGNORE statement, old data in production was dropped, and unique data copied back to the production table.
CSVs for Compliance, Permits, and SPUD dates were obtained from http://www.portal.state.pa.us/portal/server.pt/community/oil_and_gas_reports/20297 using their respectively linked utilities. Maximum date range from 01/01/1800 to 05/26/2015 was used to obtain data. Default settings for all categories were appropriate except for "inspections with violations only" for compliance, which had to be set to "no" in order to properly obtain all compliance records.
Compliance does not seem to produce results properly. Even when selecting a date range for a singular day, the error "An error occurred while executing OnInit: Date Received From should not be later than Date Received To (rsErrorInOnInit)" occurs. Otherwise queries for the data merely time out.