Assertions, Sources, Citations and Repositories

[Uncle Buddy]

Now that I'm getting data into the Treebard database directly from a GEDCOM file, it's time to take a closer look at the table schemas involved. This part of the database (see thread title) as it exists today was created a long time ago and I didn't really know what I was trying to do at the time. Here are my notes as I make plans to clean these tables up and create needed tables that are missing.

Terminology:

1:m, one-to-many relationship; a foreign key can go in the many side more than once
m:m, many-to-many relationship; two foreign key columns, either side can repeat
1:1, one-to-one relationship; best to put both columns in the same table

finding in the code: =conclusion in the GUI
claim in the code: =assertion in the GUI
conclusion: What the user decides about an event based on 0 or more assertions.
assertion: What citation(s) say about an event.
events table: should be called conclusions table
attribute: a type of event that is undated, dated with a span date, optionally dated, complete without a date, and/or better not dated at all. EG: hair color: gray. Unless it turned gray prematurely, who cares that it turned gray?

Create the following tables or correct existing tables:
1. Sources and citations: 1:m `source_id` is FK in `citation` table OK
2. Repositories and sources: m:m make junction table `sources_reposits`
3. Findings and claims: 1:m `finding` is FK in `claim` table
Why isn't this a many-many relationship?
claim: Bob was born in 1888 in the Sahara Desert. > 1900 census
claim: Bob was born in 1898 in Africa. > biography
finding: Bob was born btwn 1888-1898 in Sahara Desert, Africa > 2 claims
If m:m, then you could have 2 conclusions for the same event. This is no longer a conclusion and defeats the purpose of separating assertions from conclusions. So it looks like death and birth are not the only events which can't/shouldn't be repeated in a person's events table. Unlike death and birth, I don't think there's a way to enforce this, because two residence events could be either two different events or two versions of the same event if the user doesn't respect the notion of making only one conclusion per event. However, enforcing death and birth restriction of "one each" is easy because one person is biologically incapable of being born twice or dying twice, so there's no ambiguity and it can be enforced. Anyway, Treebard doesn't intend to become enforcementware. But discouraging the same event from appearing multiple times in the same events table is one of Treebard's reasons for existing.
4. Citations and claims: m:m make junction table `claims_citations`

I read the other day, I think it was on the FHISO website, that modeling assertions in genieware is an abstract prospect. True, if an assertion is "Bob was born on the back of a camel somewhere in the middle of the Sahara Desert in about 1897 or so while his parents were racing from one oasis to another, blah blah blah" then that is not genealogy, it's a note. But in Treebard the conclusion and the assertion can be recorded pretty much the same, with each column containing a single part of the assertion made by a source. Is it double the work? Well, if there are fifteen different assertions about a single event, it's considerably more than double the work. But Treebard is not slanted toward either evidence- or conclusion-based functionality. The user who wants to make split decisions and record everything as a conclusion can do so. The persnickety genealogist who cares about the details and the sources will finally have a place to store his data properly instead of stuffing all the good stuff into long notes. But when notes are still needed, Treebard does it right, by making it easy to link any note to any number of other elements in the tree.

I have some work to do on the old database tables before I'll be able to store GEDCOM-sourced citations in a Treebard file. This site was helpful in gathering my thoughts about the table relationships as well as confirming my suspicions that the right way to communicate between databases is with a database.

[Uncle Buddy]

I'm taking a break from the video log of me live-coding the assertions & sources table to re-evaluate some things I've been assuming all along. There are two main ideas.

1) Many-to-many tables could all be combined into one junction table called links_links. This was brought home yesterday when I accidentally linked a source and repository by putting foreign keys source_id and repository_id in a new row of links_links. I had forgotten that I'd recently made a new table sources_repositories. But why did I bother to make it? So I just deleted the table. Similarly, it seems like findings_roles, claims_citations, images_elements, and other many-to-many tables should just be deleted, and the data moved to links_links. Even the rare special column that exists in these tables could be moved, maybe renamed more explicitly. For example, in places_places, there are only two columns for place_id foreign keys and a column called `period` for the span of time during which place_id2 was the parent of place_id1. What's wrong with putting that into links_links and just renaming it `parent_place_span` or something? Similarly, there's a boolean column called `main_image` in the many-to-many table `images_elements`, but other than that, all the foreign key column in the table already exist in links_links. What's wrong with moving main_image to links_links? Not that any of this is really necessary, but the existence of two possible places to do something can lead to confusion like what happened yesterday when I forgot that sources_repositories even existed since I didn't need it to record the desired link.

2) This one is a big deal. All along I've assumed that there would be an assertions table with similar columns to the conclusions table, but more of them, such as the addition of a Name column so the user would see what name was used in the source. But now that I'm facing the actual task of designing the assertions dialog, I have to ask myself what is being asserted, why, how, etc., and exactly how that relates to the conclusions table.

It no longer seems to make sense to have a row of parts to an assertion, such as a date and a place and some roles and some notes which somehow add up to as assertion. A source asserts more than one factoid, and that is more than one assertion. If you say Betty has red hair and freckles, and I say Betty has blonde hair and freckles, that's a total of four assertions.

If assertions were compounds of factoids as I'd assumed, the problem is that you might want to link part of such a compound assertion to a conclusion, but not the rest of it. So if a source asserts a name, a date, and a place for an event, it makes sense to represent this as three or four separate assertions, and then link the conclusion to only the assertions that actually support that conclusion. Otherwise, the assertions feature will create this same quandary that I'm in right now, every time it's used.

Treebard GPS is supposed to make things simpler, not more complicated, for the user.

There's also the chance that I'll discover my hypothesis is wrong. Maybe the assertion is not the natural mediator between conclusions and evidence. If that is the case, I'll revise whatever I have to, and plow on. Treebard already has plenty of reasons to exist--including my personal entertainment--and I will drop any that turn out to be unrealistic, wrong or useless.

I'm pretty sure that the next step is to try and visualize how an assertions dialog will be laid out, what it will contain, and what functionalities it will present to the user.

[Uncle Buddy]

Another thought pops up lately. What if each factoid that currently helps to make up a "conclusion" is actually itself better represented as a single conclusion. Currently there's a whole row of factoids assigned to a finding_id. (A conclusion is called "finding" in the code.)

What if that date in that conclusion row should be linked, all by itself, to asserted dates found in sources?

If that turns out to be the case, a lot of things would have to change. The SOURCES button in the conclusions table would be useless, for one thing. In fact, this might lead right back to the way that other genieware links a factoid such as a name or a place directly to a citation. But Treebard's divergence from this tradition of computer genealogy could still be to insert a number of assertions between the conclusion and the evidence.

If that turned out to be the right way to go, possibly a major rewrite could be avoided in this way: "finding" could continue to refer to a row of what would become "conclusions". The person_id and event_type could continue to be the filters which point out what the topic is, (like keys) and the other cells, such as date, place, particulars, age, roles, and notes could be the values, as in key: value pairs. Example: John Smith birth: Feb 14, 1889; Colorado; 9 months premature; 0; midwife: Flora Transcendito; note: blahblahblah. Then "claim" could be deleted from the project (we haven't really started using it yet) and "assertion" could march in and sweep everybody off their feet.

This all sounds like a pretty serious re-ordering of my thoughts, but it's fairly typical. Often it isn't till I have an actual nail in one hand and a real-life hammer in the other that is occurs to me that I have nothing to bang the nail into but air. There's a reason why brainstorming is brain storming and not brain drizzling or brain sprinkling or something harmless. Any decent brainstorm is going to shake some things up.

[Uncle Buddy]

Videos have been going up in the series on the on-camera creation of the assertions dialog, and two more will be going up later today. That's Treebard Genealogy Software at rumble.com. In preparation for the next video, I made these notes:

An assertion and a conclusion always exist separately from each other. Both are always optional. One is not the basis for the other to exist in the database. They can be linked together but it's optional, and linking the two together doesn't change either one. They can also be freely unlinked from each other without changing either one. This system lends a lot of flexibility to the assertions system, so that it serves the genealogist vs. the genealogist serving it.

The primary key, name_id, event_type_id, surety, and person_id columns are required for each row in the assertion table of the database; they can't be null. The other rows are details: date, place, particulars, age, and in each row of the database table, exactly one of these can have a value. Out of these four detail columns and any others that end up being needed, an assertion forms in one set of circumstances which could be called an event or attribute but which is actually defined by the linking of an assertion ID with a citation ID.

This discourages the linking of the genealogist's opinion about the quality of multiple assertions which are associated with each other by being in the same source. For example, the assertion made about someone's age can be nearly worthless, while the assertion made about someone's place of residence in reference to the same event and from the same source can be of high value. The value of the two factoids have nothing real to do with each other.

So it can be seen that a variety of assertions are made about possibly a variety of persons from a single source, but this only implies that all these things should ideally come together as a single event or attribute. What's important in each case is the influence that a single assertion has on a conclusion. To keep this system from getting out of hand, the single conclusions are made by the genealogist, not the sources, and because of this, it's natural to link associated conclusions together as one finding, displayed as a single row on the conclusions table, that is, a single event or attribute, whose parts are always changeable at the discretion of the genealogist.

The number of assertions linked to each row of the conclusions table and displayed as a number on the SOURCES button in each row will be the sum of the assertions linked to that row's date, place, particulars, and age columns. As yet I still have no clue how to handle rows and notes in the assertions dialog. I think they belong there but I want to get the simpler things working and then see how the roles and notes can be worked into the GUI for the assertions dialog.

I've had to overcome my own resistance to how this assertion table in the database will "look". Actually the visual feel of the database table is an illusion. The neatnik desire to see a row in the database nicely filled out has nothing to do with how database tables are designed. For example, database columns can be in any order, and requiring several-to-many null values in each row is normal. I'd always assumed that the assertion table would be very similar to the finding table, but when I went to put the hypothesized relationship of conclusion to assertion into practice, some of the assumptions I'd made proved to be invalid, as far as I can tell at this point. Fortunately I'd left the design of the assertions dialog till last, because I had many small decisions to make about how I wanted things to work, decisions which were as important as they were small. Now with the foundation of a solid design for a variety of interacting parts of the GUI and database structure, the all-important assertions functionality can be created to work with the other requirements that have become apparent in the four years that I've worked on Treebard full-time.

[Uncle Buddy]

It just occurred to me that the person_id and name_id aren't both needed, since the person_id can be got with a JOIN if the name_id is known.

It also has to be re-iterated that the conclusions table is not meant to be perfect or particularly precise. It's meant to be what genealogy is: best guesses based on relatively rare source documents. In fact, if the source documents were not relatively rare, genealogy would be boring and you'd have to pay people to do it. But since it's a treasure hunt and it's not always easy, it draws people in. It's a fun hobby.