Information theorists wonder, "What is information?"
One information theorist suggests that information is a thing. See Michael Buckland, Information as Thing.
Buckland distinguishes information-as-thing from information-as-process and from information-as-knowledge.
This line of analysis naturally suggests the question: What is the difference between evidence and information?
It is interesting to see that Buckland seems to say or suggest that is inappropriate to restrict "information" to matter that is informative; he thinks, I gather, that material that is merely potentially informative should be treated as information. Buckland's position is roughly isomorphic to my claim that matter that is only potentially relevant can count as evidence -- that, i.e., irrelevant matter may indeed count as evidence.
But then what's the bottom line? What is information? Buckland states that "the term 'information' is commonly and reasonably used to denote some population of objects to which some significant probability of being usefully informative in the future has been attributed."
Hmmm ... let's ruminate about this. (N.B. Buckland includes "events" in "objects.")
Could or should we say that "evidence" consists of objects (and events) that may be sources of knowledge about facts? Stated otherwise: Is it the case that "evidence" = "evidential source" = "spatio-temporal object or event that potentially adds to knowledge -- including partial knowledge or degrees of knowledge -- about some possible spatio-temporal event or state"?
Note this last definition still leaves the possibility that almost anything may count as evidence. (But, under this definiton, can principles of mathematics be evidence? Can a mathematician's computations or proofs be evidence? [Should they be?] If not, does the testimony of a mathematician about mathematical principles count as evidence?)