Difference between revisions of "Problem stations"

Revision as of 01:37, 28 November 2007

special cleaning solution

The cleaning solution just uses a small subset of sites including the Denali sites and sites that are often messy. Run it like the line below or see the make-clean file.

Alaska_cleaning_solution 07aug12

more commands

del_pt

If only a few points have bad pseudorange you can use

del_pt

to delete those points. Because there are so few of them I (Jeff) think it is better to remove both phase and pseudorange data for these points, as opposed to removing all pseudorange data for GPS37 using del_pcode_arc. Also, in my experience when there is a few points like this, often the phase data are bad also. So I would go back to the qm directory and use del_pt by hand to remove the points.

An example:

1448/flt> allbadp
07oct08gmas____u0.postlog
  -119496000.00  GMAS  GPS37  8-OCT-2007 15:44:46.0000 110
  -119495000.00  GMAS  GPS37  8-OCT-2007 15:49:46.0000 110
  -119494000.00  GMAS  GPS37  8-OCT-2007 16:14:46.0000 110
  -119494000.00  GMAS  GPS37  8-OCT-2007 16:09:46.0000 110
  -119494000.00  GMAS  GPS37  8-OCT-2007 16:04:46.0000 110
  -119494000.00  GMAS  GPS37  8-OCT-2007 15:59:46.0000 110
  -119494000.00  GMAS  GPS37  8-OCT-2007 15:54:46.0000 110
      447900.00  GMAS  GPS30  8-OCT-2007 15:44:46.0000 110
      435303.00  GMAS  GPS54  8-OCT-2007 16:09:46.0000 110
      434522.00  GMAS  GPS24  8-OCT-2007 16:14:46.0000 110
      433306.00  GMAS  GPS24  8-OCT-2007 16:09:46.0000 110
      433262.00  GMAS  GPS24  8-OCT-2007 16:04:46.0000 110
      432806.00  GMAS  GPS36  8-OCT-2007 16:04:46.0000 110
      432401.00  GMAS  GPS36  8-OCT-2007 16:14:46.0000 110
      432376.00  GMAS  GPS36  8-OCT-2007 15:59:46.0000 110
> cd ../qm
> gunzip *08gmas*
> del_pt *08gmas* GMAS GPS37 "8-OCT-2007 15:45"
#  and repeat del_pt for the other 6 points

pppclean

If a station solution seems to have quite a few cycle slips, then I (Jeff) would run

pppclean

on the command line to let it do its work automatically. You might have to repeat pppclean again.

An example:

pppclean 07oct08gmas____u0

It is perfectly OK to do this, because this is what autoclean would have done if it had not been messed up by a few bad points. But if there are small cycle slips or a few outliers left, pppclean will not find them (it uses a 10 cm tolerance for jumps in the residuals).

assignments of problems

Here should be an overview/short description about the problems the following stations have/had.

299C huge outlier found in the time series plots
AC16
LOGC is historically noisy (bad sky view)
PETP has episodic problems with severe RF interference
ZECK has a history of being a bad station from time to time

299C

There was a huge outlier found in the time series plots. I found the real problem. Look at these lines from the .pfiles file:

2005.5753 299C . 217.924221759 64.028943706 745.1622 2.8 3.0 5.7 -0.054 -0.081 0.264 
/gps/analysis/1333/post/05jul29alaska2.0_nfxigs03.poscov
2005.5781 299C . 217.924221808 64.028943690 745.1676 2.8 3.0 5.7 -0.056 -0.093 0.275 
/gps/analysis/1333/post/05jul30alaska2.0_nfxigs03.poscov
2005.5808 299C . 217.936350543 64.031716461 726.6352 2.7 2.9 5.5 -0.058 -0.086 0.296 
/gps/analysis/1334/post/05jul31alaska2.0_nfxigs03.poscov
2005.5836 299C . 217.924221708 64.028943689 745.1762 3.0 3.1 5.9 -0.047 -0.080 0.259 
/gps/analysis/1334/post/05aug01alaska2.0_nfxigs03.poscov
2005.5863 299C . 217.924221781 64.028943654 745.1625 2.7 2.9 5.5 -0.049 -0.101 0.278 
/gps/analysis/1334/post/05aug02alaska2.0_nfxigs03.poscov

Clearly one does not match: 05jul31. And looking in the directory for that week it is easy to see why:

EVEREST 1334/flt> ls -l *alaska*point*
-rw-rw-r-- 1 akda gipsy     573 2007-09-12 04:55 05aug01alaska2.0_nf.point.gz
-rw-rw-r-- 1 akda gipsy     688 2007-09-12 18:51 05aug02alaska2.0_nf.point.gz
-rw-rw-r-- 1 akda gipsy     604 2007-09-13 06:36 05aug03alaska2.0_nf.point.gz
-rw-rw-r-- 1 akda gipsy     586 2006-09-12 12:06 05aug05alaska2.0_nf.point.gz
-rw-rw-r-- 1 akda gipsy     268 2006-09-12 22:06 05aug06alaska2.0_nf.point.gz
-rw-rw-r-- 1 akda gipsy 1963702 2007-09-13 17:50 05jul31alaska2.0_nf.point.gz

Several days of this week were recently rerun for some reason (possibly to add data, or some minor cleaning done). But one of them contains some awful data. Because the solutions in this week were either run back in 2006 or just recently, my guess is that this is our old friend WES2 messing up another day. There is a point positioning solution in the directory that confirms that. So when WES2 is fixed and the solution is rerun, all should be well again.

What causes problems like this? I went back to the original qm file and reran the point positioning solution, and the bad data are quite clear. Do this:

cd /gps/data/1334/flt
postplot WES2 ALL *31wes2*fit -d 110

FYI, all the time series plots for Alaska were clean of serious outliers when I left. So if you see others like this on any station it probably means that a solution rerun since then has a serious problem. Also, if you are rerunning any solution that was last run before February 2007, it is worth your time to check whether this is one of those days when WES2 had bad data, or whether any other station where the data were added recently might have a problem. This should often tell you:

ls -l *____*point*

AC16

LOGC

LOGC is historically noisy, which is easy to understand if you have ever seen it (bad sky view). Also, once it starts to snow, the noise level at LOGC rises. So expect to see more bad points then -- we might see some of its effect at the very end of the data we currently have. I have not seen any solution to the problem of LOGC other than deleting all the outliers. It's a pain.

PETP

The site PETP has episodic problems with severe RF interference. A good example is 07oct23 in week 1450 (files also copied over to the /gps/data/bad_examples/flt/ directory).

postplot PETP ALL *23petp*fit 

If you look at this plot you will see a period of significantly elevated residuals from about hour 26 to hour 34. Then the residuals get small again and go back to normal. I have tried many times before to fix up PETP files that look like this, and the best solution is to remove all the data from this window. If you add -xel to the postplot line, you can see that there is almost no elevation dependence of the residuals, and if you add -xaz instead you can see that the largest residuals are concentrated in a certain range of azimuths.

There is a later period of large residuals, but you can see that the residuals for GPS35 form a stair-step pattern. I count 4 small cycle slips on that satellite over about a 3 hour span. I would figure out the times and run add_amb manually to add ambiguities at those three points (short_hand will not handle this well because the times of the slips do not line up with times of large residuals, so do it by hand), and then after doing that and removing the first bad window I would run the point position again and see what is left).

It is possible that PETP will be like this most days for a month or so, then will suddenly stop and go back to normal. You will not see these problems at the same time of day -- it depends on when they turn on whatever powerful transmitter is causing the interference.

ZECK

week 1433, day 29

Final assessment of ZECK data from this day: it is junk. I'll elaborate on my assessment so that you can learn how I approached it. Part of what contributes to that is that I know ZECK has a history of being a bad station from time to time, although not every day by any means. Maybe it has some intermittent RF interference? I went back to the original file, and it had some enormous pseudorange residuals for GPS37. These would have caused autoclean to mess up the file, and I am pretty sure that these bad pseudorange points would have still been obvious in the point positioning solution left after autoclean. If so, the first thing you should have done was go back to the original file, rerun pppsolve., and then deleted the bad pseudorange for GPS37 using del_pcode_arc.

Then I reran it and got something that looked a lot like what you had in your file after more cleaning, just not quite as bad. I'll leave those files there for now so you can look at them and then delete the files (also move the qm file to a bad subdirectory). What I saw was that some satellites looked OK, but several of them had excursions of up to 20-30 cm magnitude. If you look at some of the worst ones (see list below), they do not look normal at all. The later part of ZECK-GPS41, for example, looks like a staircase going down to the right. That might be a sign of several small cycle slips in the data. Looking at other satellites, I can see 5-10 cm jumps all over the place.

maxi 07jun29zeck____a0.postlog | head
         -25.06  ZECK  GPS29 29-JUN-2007 17:04:46.0000 120
         -24.61  ZECK  GPS29 29-JUN-2007 17:09:46.0000 120
          20.00  ZECK  GPS41 29-JUN-2007 17:49:46.0000 120
          19.29  ZECK  GPS56 29-JUN-2007 10:49:46.0000 120
         -18.87  ZECK  GPS40 29-JUN-2007 10:39:46.0000 120
          17.94  ZECK  GPS46 29-JUN-2007 08:14:46.0000 120
          17.90  ZECK  GPS27 29-JUN-2007 08:39:46.0000 120
         -17.86  ZECK  GPS34 29-JUN-2007 07:54:46.0000 120
         -17.26  ZECK  GPS40 29-JUN-2007 10:44:46.0000 120
         -17.26  ZECK  GPS32 29-JUN-2007 08:39:46.0000 120

What this means is that maybe, after a LOT of work, you might be able to rescue this file. And unfortunately the problem is nto confined to just one small length of time, but is spread out over most of the day. When I see this with a station that has a history of periods of bad data, it is usually a sign that it is time to throw it out. Or perhaps to try once or twice and see if it gets better, and throw it out if it does not. I'm guessing that you went several iterations and didn't get anything that looked right, so certainly you will be justified in moving this file to the bad/ subdirectory. Be sure to create the qm/bad subdirectory if it does not already exist before you do a mv *29zeck* bad, otherwise you will just rename the file.