Return to HomePage
Return to the Index of VSNET discussion topics
I'm gonna go out on a limb here and say: CCD observations that have large errors are (most likely) systematic. That is, a reproducible event or error is involved, such as a bad comp star, bad image reduction, etc. In theory, if one makes a note of all related criteria, one can always re-reduce a CCD observation based on better information and revise the estimate accurately. CCD observations are, in this way, not subjective. You can screw up your process but it is hard to introduce subjective error.
Visual observations, on the other hand, rely solely on the experience and expertise of the observer and are much more subjective. The visual observer writes down how an estimate based on the way it looks to them at that moment. There could still be some systematic error, if a bad comp star is used, for example, but there is additional uncertainty based on the human being. You could go back and revise a visual estimate based on the systematic variables, such as comp stars, but you have a certain amount of uncertainty that is uncorrectable.
So if two CCD observers have very different measurements, you can probably figure out why. If two visual observers have two very different estimates, you *may* be able to figure out why, if a bad comp was used or something, but you may not. One human saw one thing and the other human saw another. There is no reproducible way to unravel the difference.
This is not to say that CCD observations are more accurate, but certainly in theory at least, the uncertainty is less subjective.
Michael Koppelman
There are two main sources of errors: random errors and systematic errors. CCDs are likely superior in terms of random errors, if the other conditions are the same. Or one can use the terms like internal errors and external errors. CCDs are superior in reducing internal errors (with which we can make fine time-series analysis of low-amplitude variables). However, external errors are more difficult to reduce, and in order to do so, one will require high quality control (e.g. checking consistencies between different nights and images), which does not seem to be reasonably performed by every CCD observer. This would be one of reasons why CCD observations of individual variable stars just as in the procedure as in visual observations have not been surprisingly accurate as is expected from small internal errors in time-series photometry.
The other thing is more human. It seems to me that the fraction of careless mistakes is not reduced in CCD observations (or even more abundant in CCD reports). These careless mistakes include incorrect identification of comparison stars or even objects, bad calculation, poor use of reduction software, error in copying data from the software output, date error, and many, many. It sometimes looks like that CCD observers tend to think there primary aim is to make observation, but not to make accurate reports. Some people don't care if the variable is calculated to be several magnitudes different on subsequent nights, and don't care if they confuse a 10-th mag star with a third mag star. It may have partly due to the fact that CCD observation and reduction are usually more time-consuming than visual observation, and people tend to make mistakes and dismiss doubts in fatigue.
Not all CCD observers store their CCD images; some people indeed delete the images after sending reports. Not all errors in CCD observations can be re-reduced as in theory.
Regards, Taichi Kato(vsnet-chat 5972)
I'm sure this is true but it sure strikes me as odd. I *enjoy* working with my data so much that I take my time with every observation. I wonder how often the careless mistakes you mention are a result of people processing large numbers of images automatically. TASS, for example, does this but they do so with very high precision. (They don't submit their observations to the databases that I'm aware of, either.) Perhaps some of the careless mistakes are survey-type of activities without the care that TASS (and others) use?
For non-survey type CCD observations it just baffles me why anyone would submit any observations that were not as careful and as correct as possible. Observations are worse than useless if you don't take care in the reduction and reporting.
Michael Koppelman
We have no information about the frequency of errors in TASS observations. The careless mistakes are mostly by other survey-type observations, and some CCD snapshot observations. People involved in CCD time-series observations also sometimes submit curious data (time error, comparison error etc.). There was a well-documented error in the former version of MuniDOS (see vsnet-alert 3979, http://www.kusastro.kyoto-u.ac.jp/vsnet/Mail/alert3000/msg00979.html), which was used by a number of observers. I know numerous examples of errors in heliocentric corrections (this is the reason why the VSNET only accept observations without heliocentric corrections).
A number of star mapping programs have been know to have bugs in their variable star positions (possibly from errors in B1950.0 to J2000.0 corrections, other format problems etc.), which were sometimes reflected on snapshot CCD or photographic reports (but similar errors are rarely met in visual reports, this is probably because visual observers have more experience to carefully check the field identification).
It even seems to me that new errors are continously injected every time a new program or a version is released. It would unavoidable for every program to have some problems, but it sometimes looks like to me that CCD observers tend to preferrably or selectively use most unreliable software among various selections. X-) The natureal result is that database managers or data compilers, not observers(!) or software programers(!), are obliged to continously chase after errors or spend much time to identify software problems. I would bet "careless" researchers may have used these unreliable data without any doubt, and may have published solid papers on them.
Regards, Taichi Kato(vsnet-chat 5979)
Things would have been better if every observer strives for accuracy... It doesn't seem to be that every observer wants to turn in accurate data. The observer's preference on accuracy, number of reported data, reduction of efforts, human intervention etc. seems to be very different between observers.
Regards, Taichi Kato(vsnet-chat 5988)
> It even seems to me that new errors are continously injected every time > a new program or a version is released. It would unavoidable for > every program to have some problems, but it sometimes looks like to me that > CCD observers tend to preferrably or selectively use most unreliable software > among various selections. X-)This is an intriguing comment. Most professional astronomers prefer to avoid endorsing or recommending against available software, and yet here we have the possibility of some important knowledge.
Dr. Kato, which software do you believe is "most unreliable"?
-Lou Cohen
Every software is as reliable as each user can make it to be.
Whenever one assumes blindly that all the softwares does what one thinks they do, then the person is one step closer to shooting his/her own feet. So don't assume that there is such a thing as the "most unreliable" software. More often, many "software" problems are due to user errors.
In short, everyone should have some simple way to test and see if your "software" works the way you expect it to be. If not, ask developers about it.
Cheers, Bish -- "Bish" K. Ishibashi, Ph.D.(vsnet-chat 5990)
User errors are a serious cause of the problem. Even if the software gives correct results, an error in identification of the variable star will not be recovered even by careful programming of the software.
Even disregarding such a cause of errors, some software more frequently gives incorrect results than others. If there is a JD to HJD (heliocentric conversion) error, most users will not be aware of the error until he or she compares the results with other software, or finds a deviation from the ephemeris -- but such lucky things rarely occur, and most of the problems are more subtle, or more difficult to identify. If some program produces such difficult-to-understand problems more frequently than others, it would not be a bad approximation that such a program would produce more frequent problem in other circumstances.
The problem in software is that it is becoming incresingly difficult to fix bugs. It is a good approximation (from a view of information technology) that the frequency of bugs is proportional to (complexity)^N, where N is larger than unity. This is well-known that this scaling law determines the maximum reliable size of software, at which the introduction of new bugs (proportional to (complexity)^N) becomes comparable to removal of bugs (proportional to ~complexity or less). The present complexity of software already exceeds that of user's ability to regulate the occurence of bugs. Bugs are increasing more rapidly than users can identify them. Think of the earliest bug in Pentium. That was a very simple bug (incorrect arithmetric), but it took a long time before the problem was recognized.
So it is virtually impossible to expect users to make such tests. Even if a user would have motivation to make tests, either most the information of the software is insufficient or the user lacks the necessary skills. Even in the luckiest cases when the problems are identified, developers are usually very busy and are usually reluctant to fix bugs in already released software, and are usually more rushed to continue developing and releasing new software.
Regards, Taichi Kato(vsnet-chat 5992)
I con't tell which is the most unreliable software ;-), but know some software is more unreliable than others. If you have a chance to look at the source codes of some software, you can easily recognize how "many programs are unreliable". But this is usually hidden from users. (This is one of the reasons why the VSNET makes the source codes of the software open to the community. The same is true for problems in variable star charts.)
So, in many cases we need to speak of a certain software from a purely phenomenological viewpoint. I once banned our local users from using some chart plotting software because of a high frequency of errors in variable star positions, but I recently see the same name of software frequently in some or other mailing lists. And what is worse, similar errors (possibly from a different origin) are found in other software. We have no good idea to regulate such errors. Every time new chart plotting software is released, a similar thing can continue forever. My recommendation is, thus, not to use commercial chart plotting software for variable star observing!
Regarding the MuniDOS error (I don't want to mean that this software was the most unreliable one, but I cited this because we could clarify the source of the error) which I referred to, even a community led by a certain group of professional astronomers is proven not to be an obstacle against promulgating that software-borne problem before the software was widespread; this software was even endorsed by a group led by professional astronomers. It took a long time before the problem was recognized only when some data were reported to VSNET with this software, and we noticed some inconsistencies between the reported data. I'm not certain whether the old (wrong) version of this software has been perfectly removed from the entire world. There would be a good chance that some fraction of the presently reported observations comes from the old version of this software, and there is no way, even if the AAVSO HQ works hardest, to recover the correct data from the wrong data.
Regards, Taichi Kato
Return to HomePage
Return to the Index of VSNET discussion topics
Return to Daisaku Nogami's page
