Correction Tables for Sextant Altitudes

The Altitude-Intercept method now generally used for Celestial Navigation, is based on the comparison of an "observed altitude" (Ho) with a related "computed altitude" (Hc).
The computed altitude is based on a mathematical model implying a number of conditions and assumptions some of which are:

  • The celestial bodies have no physical dimensions and observations are made from the center of the Earth.
  • Light rays from celestial bodies come from infinitely far away and hence reach the Earth parallel to one another.
  • The Earth has no atmosphere and air has no index of refraction.

These assumptions are not full-filled for observations made in the physical world. In order to put the mathematically computed altitude and the "real world" observed altitude on an alike basis of comparison, some "corrections" must be applied to the measured sextant altitude.

These corrections are performed on the measured sextant altitude according to the following scheme:



  Hs     __ ° __'_                Sextant Altitude
  IE   ±      __'_                Index Error
  dip  -      __'_                Dip Correction
       _____________
  Ha     __ ° __'_                Apparent Altitude
  Refr -      __'_                Refraction
       _____________
         __ ° __'_                
  Prlx + __ ° __'_     HP __°_    Parallax Correction
  SD   ±      __'_     SD __'_    Semi-Diameter Correction     
       _____________              (lower limb:+ /upper limb:-)  
  Ho     __ ° __'_                Observed Altitude

The sextant altitude (Hs) corrected for index error (IE) and dip correction (dip) is called "apparent altitude" (Ha). This value is used as entry for further corrections (refraction (Refr) and parallax (Prlx) ).



The values for "semi-diameter" (SD) and "horizontal parallax" (HP) must be obtained from the Nautical Almanac. The required values for the correction of dip, refraction and parallax can be found in the pre-compiled correction pages (125 Kbyte PDF file), or they can be obtained from the interactive tables below.
The pre-compiled PDF file can be viewed and printed e.g. with the free tools "Evince", "xpdf" or "Acrobat Reader".



 


Correction Table for Dip

The "dip correction" compensates for the fact that observations made by a navigator through a sextant are not performed on the "level of the horizon". The difference between the visible horizon - the line where the sky meets the sea - and the geodial horizon - the plane perpendicular to the zenith line in the location of the observer - is called "Dip of the Horizon". This "dip" is not only caused by the elevation of the observer's eye above the plane of the geoidal horizon (He) but also by atmospheric refraction.


 
Table
 

The higher the observation point (sextant telescope) is above the sea level, the further "away" the visible horizon seems to wander. Due to this effect, the measured altitude values are to high. So "dip corrections" are always negative.
It is important to use the correct value for He and not just a coarse estimation. Using a He value that is 2m off, will result in a LoP that is one mile off!


The correction for dip and the distance to the horizon can also be calculated with the following formulas:

  Correction_for_Dip ['] = 1.760 ['/sqrt(m)] * sqrt (He [m])

  Distance_to_the_Horizon [Nm] = 2.075 [Nm/sqrt(m)] * sqrt(He [m])

These equations are empirical and include the effects of the curvation of the Earth and atmospheric refraction.

 

 

Correction Table for Refraction

Refraction is an optical characteristic of the earth's atmosphere. Light coming from outer space is deflected to the earth's surface when it travels through the atmosphere. This is caused by the increasing density of the air towards the earth's surface.

The result is that at sea level a celestial object is seen at a different angle as it would be seen at the same location without the presence of the Earth's atmosphere. Corrections for refraction are always negative and they primary depend on air temperature and on barometric pressure.


 
Table
 

Since the atmospheric processes involved are very complex and air masses with different temperatures are continuously stirring the troposphere, it is very difficult to obtain good models describing refraction (especially for low values of altitude). For this reason, a "Line of Position" obtained from an Altitude lower than 15° can be considered to be rather inaccurate.


The largest corrections for refraction occur at high pressure and low temperature. Under these conditions the "air density" and thus refraction will obtain the highest value.

NOTICE: The barometric pressure used in these tables is the "real" atmospheric pressure at the location of observation, not the barometric pressure reduced to the sea level. Therefore, the pressure range of this table will not be sufficient to cover locations significantly above the sea level.

For altitudes larger than 15° the approximate correction for refraction (Refr) can be calculated from the apparent altitude (Ha):

  Refr ['] = 0.97127 ['] * tan (90°-Ha [°]) - 0.00137 ['] * tan3(90°-Ha [°])

 


Tables for Parallax Correction

Parallax correction of apparent altitudes (Ha) is needed for celestial bodies which are "close" to the Earth. This applies to the Moon, which may have a parallax angle of more than 1°; but also the altitudes for the planets Venus and Mars may be corrected for parallax effects to obtain the best possible results.


 
Table
 

The horizontal parallax only depends on the radius of the Earth (which is constant) and and the distance of the celestial object to the Earth. Since the distance of celestial objects to the Earth changes very gradually, the horizontal parallax does not change significantly during one day. Therefore, the Nautical Almanac records the "horizontal parallax" angle (HP) only for a for a small number of distinct hour values each day. The HP value corresponding to the hour nearest to the moment of observation can be used without interpolation.

The horizontal parallax angle (HP) and the "apparent altitude" corrected for Refraction and Semi-Diameter (SD) are used as entries for the Parallax Correction Tables.

A detailed presentation of the geocentric parallax and how it is taken into account while performing celestial observations, is summarized in the "Notes on the Calculation of Geocentric Parallax".




Cover  <<  Sail Away  <<  Celestial Navigation  <<  . last updated: 08-Apr-2003