I'm conducting a thorough audit of all the applications I've been developing to feed my research, trying to integrate, link, and add functionality to them. Again, I rely heavily on solutions like DeepSeek and Claude for code review and generation, but always ground them in a detailed understanding of what I want to achieve from an engineering standpoint and the authoritative sources involved. I build the logic, search for the appropriate endpoints, and thoroughly test the prototypes.
And when I say thoroughly, I mean thoroughly. I'm very demanding about quality and achieving optimal performance in my endeavors, whether technical or intellectual. With respect to the first version, several important changes affect both the logic and the scope of the search.
It is no longer simply a matter of searching for contracts and grants using keywords related to Cuba.
After extensive work with the data returned by USASpending.gov and different filter combinations, I realized the need for some independent passes.
The first two are the basic ones:
Awards with Cuba as the primary place of performance.
Awards retrieved through keyword searches related to Cuba.
This separation matters because, when combined in the same USASpending query, I would essentially retrieve awards satisfying both conditions, rather than awards satisfying either one. So I run them independently and merge the resulting universes afterward, removing duplicates. This may sound like a small implementation detail, but it was one of the first important changes in how I conceptualized the tracker.
The next step was to search within grants associated with the CFDA —or ALN, under the current terminology— programs I'm interested in:
19.345
19.750
98.001
Relying on this critical endpoint, among the grants associated with these programs that did not match the first two basic passes, I search for awards containing an opportunity number that matches an identifier from the NOFOs tracked by another application I have also reformulated.
This is because, in some cases, the only reference connecting an award to the Cuba-related, 1996-born program first managed by the now-defunct USAID appears in this field. In other words, the award itself may not clearly or literally reference Cuba, but it may still be traceable through the opportunity number. That makes this a fundamentally different type of search. And one that resulted in new awards being covered. And this is one of the things I find particularly interesting about working directly with the API: you can construct relationships that the web interface can't expose.
The fourth pass came from another problem I encountered while digging into the data. I also wanted to find awards with references to Cuba not in the description of the base obligation, but in subsequent modifications. The basic USASpending search endpoint works with the base description, but I found cases where a reference to Cuba was introduced in a later modification. So I added another independent pass to look for this kind of evidence. The overall logic now looks roughly like this:
PASS 1
Primary Place of Performance = Cuba
PASS 2
Keyword search
↓
MERGE
↓
remove duplicates
↓
remaining grants within target CFDA/ALN universe
↓
┌───────────────┐
│ │
PASS 3 PASS 4
NOFO match Later modifications
│ │
└───────┬───────┘
↓
MERGE
I find these additional passes critical because they give me a much greater confidence level that I am exploiting as much as possible of what the USASpending API can provide. These two passes are optional in my frontend, since they are more expensive and computationally demanding than the basic searches.
Contracts are a bit different. After studying the many records returned by the searches, I restricted the universe using the North American Industry Classification System (NAICS).
I currently focus on these top tiers:
51 — Information
54 — Professional, Scientific, and Technical Services
I also exclude additional lower NAICS levels, as well as certain offices and federal accounts. The purpose is not to claim that every contract belonging to those NAICS categories is relevant. Instead, these filters let me remove a huge amount of noise from a universe that would otherwise be harder to inspect manually. And that makes a substantial difference.
For grants, on the other hand, I apply another structural filter when constructing the universe that feeds passes 3 and 4.
At the moment, the TAS whitelist is:
072-1037 — Economic Support Fund
072-1021 — Development Assistance
072-1027 — Transition Initiatives
The idea is to prevent grants associated with unrelated federal accounts from reaching the more expensive subsequent passes.
The other major change is the possibility of persisting on disk the decisions I make regarding matches between USASpending Award IDs and the additional_award_identification field in the FAC.gov API.
This is where things get particularly interesting.
I don't assume that the relationship between the two datasets will always be an exact string match.
The tracker therefore tries several increasingly flexible ways of finding a corresponding FAC record.
Conceptually:
Exact Award ID
↓
Exact ID + UEI validation
↓
Award ID suffix + UEI
↓
Award ID suffix + organization name
↓
Organization name + Award ID
The first attempt is obviously the strongest: look for the complete Award ID in additional_award_identification.
Then I use the suffix of the Award ID --its last five digits-- together with the recipient's UEI, or eventually the organization name. The important point is that the method by which the candidate was found is preserved. A suffix match is not treated as if it were an exact identifier match. And an identifier match is not automatically equivalent to a manually confirmed relationship.
I preserve enough information about the matching process for me to decide what those matches actually mean. I can also explicitly dismiss a candidate. The system therefore keeps two different things apart:
What the API returned
+
What the matching logic found
+
What I manually confirmed
+
What I manually dismissed
Finally, I integrated my independent FAC.gov award searcher into this tracker, because that was simply the logical thing to do.
There is a subtle but important difference here.
If I only use FAC to look for records corresponding to awards that I have already discovered in USASpending, I am assuming that USASpending will always give me the initial lead. So I wanted another, completely independent path:
search FAC.gov directly for potentially relevant awards, without starting from a known USASpending Award ID.
The independent FAC search starts from reports associated with the relevant cognizant agencies —the former USAID and the Department of State— and then works within the same CFDA/ALN universe I use in USASpending.
The resulting candidates are then filtered using Cuba-related signals.
So FAC is no longer merely a secondary source for validating or linking USASpending records.
It also becomes an independent discovery mechanism.
I am still working on the traceability between the descriptions of the different award modifications and the federal_program_name field in FAC.
I'm not expecting to find a huge amount of "gold" there, for several reasons, but I think it is worth exploring before closing that avenue.
I feel quite satisfied with the functionalities I've described here and that I now consider mature.
And, as you can probably infer from what I said at the beginning, mature is a condition that I take quite a long time to grant.
The result is becoming a genuinely hybrid system:
Automation
↓
expand and organize the search space
↓
Rules
↓
reduce noise
↓
Human judgment
↓
resolve ambiguous cases
↓
Persistent decisions
Automation allows me to expand and organize the search space. Rules help me reduce it. And human judgment is still necessary to resolve the cases that remain ambiguous after all the automated processing. This approach seems particularly useful when the objective is not simply to produce a list of results, but to construct a universe of evidence that can be examined, corrected, and progressively expanded. And I really think this version of the tracker is getting there.
Thanks for your reading.