Data Profiling

Data profiling in this context is the process of assembling information about a particular individual in order to generate a profile — that is, a picture of their patterns and behavior (compare this use of the term data profiling with that used in statistics or data management where data profiling is the examination of information describing the data or data set itself).

Data profiling is used in security, law enforcement and intelligence operations for a variety of applications — for example, to assess "trust" for security clearances or to grant authorization relating to a trusted system, or to identify or apprehend suspects or threats. The government is able to access information from third parties — for example, banks, credit companies or employers, etc. — by requesting access informally, by compelling access through the use of subpoenas or other procedures, or by purchasing data from commercial data aggregators or data brokers. Under United States v. Miller (1976), data held by third parties is generally not subject to Fourth Amendment warrant requirements. Private companies and private investigators can also generally access or purchase data from these aggregators.

Information relating to any individual transaction is easily available because it is not generally highly valued in isolation, however, when many such transactions are aggregated they can be used to assemble a detailed profile revealing the actions, habits and preferences of the individual.

In the past, much information about individuals has been protected by practical obscurity (a term used by Justice Stevens in his opinion in USDOJ v. Reporters Committee, 1989). Practical obscurity refers to the practical difficulty of aggregating or analyzing a large number of data points in different physical locations. In addition, information was often transient and not easily available after the fact. Further, even where data was available, correlation of paper-based records was a laborious process. Electronic, particularly digital, record-keeping has undermined this practical obscurity by making data easily available and potentially making aggregation and analysis possible at significantly lower costs.

Thus, as more information becomes available in electronic form — for example, as public records such birth, court, tax and other records are made available online — the ability to create very detailed data profiles increases and may raise concerns.