Date: November 2, 2005
Speaker: Roy Hall, CTO, — Thetus Corp
There is a common philosophy that says if you collect more data, you can distill it into more information, and that will result in greater knowledge, culminating in greater wisdom.
In modeling complex systems and evaluating policy decisions, we develop models that require data sets that establish initial conditions as well as continued data for model validation and correction. This data is largely non-textual. The quantity of high quality data required is daunting, but more daunting are the candidate data sources—both in the number of possible sources and the quantity of data.
As an example, the GOES R satellite system scheduled for deployment in 2012 will produce roughly 500,000 data products per day per satellite, and it is but one in a myriad of data sensing devices. Data collectors report that as much as 98% of the data they collect is never looked at. Clearly, having more does not translate into greater wisdom unless we can effectively use what we have. Addressing this is as much a social as a technical problem.