GPS analysis system
We use a GPS data analysis system based on the GIPSY software developed at JPL. Most of the GIPSY programs are called by shell scripts written by Jeff Freymueller. Using these scripts, we can analyze a large amount of data either as part of network solutions or in Precise Point Positioning (PPP) mode.
Contents
Where do you put RINEX files?
RINEX files should be put into the hopper, $RAWDATA/hopper. What, you don't have RINEX files yet? See RINEXing. Once files are in the hopper, you can either let the first processing stages happen automatically overnight (see next section), or run the autofront and autoclean programs manually.
What happens automatically?
Quite a lot.
Autoftp runs every night beginning at 6pm local time and fetches data files. These are placed into the hopper ($RAWDATA/hopper), a directory where all data files are put for entry into the system and processing. Autofront then runs at midnight to process all files in the hopper, including any placed there manually (from campaigns, for example). Finally, autoclean runs at 4am local to carry out automated screening for cycle slips and other bad data.
Autoftp
Autoftp is an efficient data-fetching tool that uses wget to automatically get data from any of several internet GPS data archives. It reads a list of desired sites from a request file, which contains the date in the filename, and attempts to find and download data from as many sites as possible. It is intended to run automatically on a daily basis under cron, and when acccompanied by another simple program to generate a standard request file every day, it can easily fetch a standard set of sites on a daily basis for analysis. Because it keeps track in the request file of sites that it has found already, autoftp can be run multiple times with the same request file and it will not repeatedly fetch data. This is ideal for the real world, in which data from some sites are available rapidly while data from other sites may require many hours or days to become available.
Autofront
Autofront is a script intended to run under cron that carried out the initial "front end" processing on a set of GPS data files. When executed, it will process all files in the hopper directory, and will place each resulting qm file into the appropriate week directory.
Autofront runs the following steps 1. Checks on the validity of RINEX file and repair of some common problems. 2. Depending on receiver type, clockprep -fixtags 3. (optional, presently not default) PhasEdit 4. ninja
Autoclean
Autoclean carries out automated cleaning of cycle slips, based on point positioning solutions. It is quite effective and at present it rarely misses cycle slips unless they are smaller than its minimum tolerance (10 cm). Autoclean operates on an edit-request file, which contains the name of the directory (week directory) and a list of qm files that need to be cleaned. It will clean all files on the list as long as orbits and clocks are available, and it marks off files that have been cleaned so that it can safely be run multiple times.
Autoclean operates in an iterative mode. It's zeroth iteration is to do a pseudorange-only solution and identify and delete extremely bad pseudorange data. In this step it uses a tolerance that catches only grossly biased data. (Explain it). It then carries out 1 or more iterations of screening the phase data. In each iteration, it uses postbreak to identify discontinuities in the residuals of a point positioning solution. Postbreak is run with an adaptive tolerance (minimum 10 cm), and it is critical that my slightly modified version of postbreak be used. If any cycle slips are discovered, they are flagged and another iteration is run. Autoclean runs a maximum of 4 iterations on the phase data.
Where do the data files go?
Data files from each station are stored in the QM format that is native to GIPSY. QM files (and all other) files are stored in directories by GPS week. For each week directory there are several subdirectories; qm files are stored in $ANALYSIS/wwww/qm, where wwww is the 4 character GPS week number (with a leading zero if needed).
Running Solutions
Solve, a very flexible script. (link to detailed help)
Philosophy of solve
Subnet files and campaign files
Standard solutions
(text of standard_Alaska_solution)
Running several days at once: make-make-flt and make-alaska
(text of a standard make-alaska file and variant)
Running several weeks at once
(text of sample rerun-* file)
Cleaning Solutions
Initial Explanation of terms
Expected residuals from a clean solution
Automated screening: postfit, the point file, postbreak
Checking for bad pseudorange data: badp, allbadp
Removing biased pseudorange data: del_pcode_arc
Automatically identified cycle slips: breaks
Quickly scanning through residuals: short_hand
Limitations of short_hand
Manually checking residuals and fixing problems (link)
Procedure for Running Solutions for a week for the first time
A few special things need to be done the very first time solutions in a week are run. First, you need to make up a script to run all days of the week. This may need to be edited if the JPL final orbits are not available at the time. The standard_Alaska_solution uses the non-fiducial orbits and thus requires that the final JPL orbits be present. If they are not, you can run rapid_Alaska_solution instead.
Then, the log files from autoclean should be moved away to a subdirectory, and any problem stations identified by autoclean should be checked. Then, you are ready to run the solutions.
First, make a script to run solutions. For example, to make a script to run all days of the week for the Alaska solution:
cd $ANALYSIS make-make-flt 1381 cd 1381/flt vi make-alaska # Edit the file if needed so that you are ready to run the rapid solutions. cat make-alaska #!/bin/csh -f # setenv CAMP $ANALYSIS/1381 # #standard_Alaska_solution 06jul01 #standard_Alaska_solution 06jun25 #standard_Alaska_solution 06jun26 #standard_Alaska_solution 06jun27 #standard_Alaska_solution 06jun28 #standard_Alaska_solution 06jun29 #standard_Alaska_solution 06jun30 # rapid_Alaska_solution 06jul01 rapid_Alaska_solution 06jun25 rapid_Alaska_solution 06jun26 rapid_Alaska_solution 06jun27 rapid_Alaska_solution 06jun28 rapid_Alaska_solution 06jun29 rapid_Alaska_solution 06jun30
The script make-make-flt finds all unique dates for qm files in that week's directory, and uses that to generate the script, so if you run it before the end of the week you will get a partial script. If the final JPL orbits are not yet present, you will need to edit the script to change "standard" ro "rapid". Or better yet, copy all the lines and comment one set out, then modify the others to read "rapid_Alaska_solution <date>".
Next, make a subdirectory called logfiles, and move all of autoclean's logfiles into it. You will need to use two separate mv commands because the list of files is too big for one.
mkdir logfiles mv *____*.i1* logfiles mv *____*.i* logfiles
Now look for a file called make-problems, which lists all files that autoclean had a problem with. Sometimes these files are almost clean, but sometimes they are full of junk or horribly mangled by the automated editing. There should be PPP solutions already run for these files, so they are ready to eb checked.
Now run the solutions. The first time you run the solutions, look at the residuals very carefully before trying short_hand. Uncompress the postlog, postfit and postbreak files, and then use allbadp to check the pseudorange and allbreaks to check for major cycle slips.
A very common problem is that for several stations per week, there will be one satellite arc of pseudorange data that all have residuals of roughly 2000 cm. If you see these, don't delete the data, but instead run del_pcode_arc to remove only the pseudorange data. I am not sure why these show up, but it could be either a hardware channel bias or a pre-processing glitch. They happen much more often with Ashtechs than any others, and are particuarly common with the US Coast Guard CORS sites. In the qm directory,
del_pcode_arc *02gus2* GUS2 GPS41
If you just run short_hand without looking, it will probably either throw out all the pseudorange for one site, or delete a lot of data (phase and pseudorange) where only the pseudorange needs to be deleted. So don't do that. Then I delete a batch of bad pseudorange data to get the number of pseudorange outliers under control for the next run.
cd $ANALYSIS/1381/flt gunzip *alaska*post* allbadp allbreaks # Based on this, run del_pcode_arc as above, and add ambiguities manually if needed. # delete_allbadp 50 # This creates a file called delete. vi delete # Remove lines for any points for which you have already run del_pocde_arc. cd ../qm sh ../flt/delete
At this point, don't worry too much about phase outliers. Basically we are trying to get the number of pseudorange outliers down into the range where short_hand will do the right thing when we run it later. Now may be a good time to run Alaska_cleaning_solution $date, which runs a smaller and much faster solution including the sites that most often need some cleaning.
