A theory of fine-grained lineage for functions on structured objects

Lineage is the process of keeping track of the relationship between the inputs of a data processing task and the parts of the output they contribute to produce. Depending on its precise definition, lineage can be seen as a form of database provenance, a means of tracking information flow in computer programs, or be used to express causality and provide counter-examples for the falsity of a logical statement. In this paper, we establish the formal foundations of a notion of lineage for arbitrary abstract functions manipulating objects that are "composite" -that is, can be made of multiple other objects. Three definitions of lineage over functions are formally defined, respectively called explanation, participation and extraction; we then establish explanation relationships for a set of elementary functions, and for compositions thereof. A fully functional implementation of these concepts is finally presented and experimentally evaluated.