One of the major assumptions behind statistical analysis of experimental results is that actual observations fall near the "true" value according to the normal distribution (for a large set of data), which is the large-number limit of the binomial distribution (for a small set of data). This assumption is hard to justify absolutely rigorously, but seems to be true in most practical cases. For instance, this art quiz, which by rights should not produce random data, yields a beautiful binomial distribution (see Figure 1).
Probability is complicated and unintuitive. That makes it hard to understand without formal training, but also makes data hard to fake in a statistically rigorous way. For instance, this NYT article explains the Benford's law, a counterintuitive observation about the natural distribution of digits in observational data; Benford's law can sometimes be used to detect fraud in data sets.
CODATA recommended values of the fundamental physical constants: 2006: NIST best values of physical constants. This shows how difficult serious, precision measurements are and how they are all interconnected. If you glance through it you will see how often it refers to theoretical calculations (generally ending up as algebraic combinations of fundamental constants, which are the things that are actually measured), including new, improved or unverified theoretical calculations. From a practical standpoint, this theoretical error can be significant in this kind of high-precision work.
In medicine, the conclusions from many small trials can be summarized by doing a meta-analysis that includes all the results, weighted appropriately for their "quality". This article summarizes some of the issues. Don't worry about the details (Q scores and different statistical tests, for instance), but this should give you an idea of the issues involved in synthesizing the results from many experiments, conducted in different places under different conditions.