BEUDAMED: Better EUDAMED

EUDAMED. The idea behind it was so great!

Here’s the pitch: Instead of 27 medical device databases, one for each EU country, you’d only have one. So if you’re a regulatory person in the EU, no need to wrestle with 27 databases. And if you’re a country, no need to re-invent the wheel by building your own medical device database which has already been built at least 26 times before. Much less overhead for everyone!

So that’s the theory. The reality, however, doesn’t look so great.

Let’s just assume you, as a normal human being, want to perform a search in EUDAMED.

Here’s what you do:

1. Navigate to EUDAMED.
2. Hm, you have to select what you want to search for: Economic Operators (what’s that?), Devices, or Certificates. Okay. Let’s select “Devices”.
3. Hm, you are now confronted with 15 (!) input fields for your search. Looking for a specific device? Enter it in “model name”. Or wait, was it “trade name”? Or “catalogue number”?
4. You hit “search”. The search takes 18 seconds (!) to complete.
5. You get a list of results. You want to right-click on some of them to open them in a new tab. Bad luck. Right-clicking doesn’t work in EUDAMED. You can only open them in your current tab. You open each result individually.
6. You continue your search by navigating to page 2 of the results. This triggers a new search, which takes another 18 seconds.
7. You throw your computer out of the window, quit your regulatory job and vote “Yes” on Brexit.

In short, the user experience is terrible.

See, that’s the problem with “requirements engineering” / design by committee / evil waterfall development / whatever else you call it: It’s very hard to document a great user experience in Jira tickets. Like, I don’t think they added the following requirements:

• It shouldn’t take a mind-boggling 18 seconds to perform a search.
• It shouldn’t confuse the user by offering 15 separate input fields.
• Right-clicking on links should actually work.
• It shouldn’t suck and make people throw their computer out of the window

But these are just the outward-facing problems we’re seeing. The underlying technical problems look even worse – my best guess is that their development process is inherently broken, their best software developers have left and now they’re left with a bunch of junior engineers who are not fit for the job.

(I’ve appended more technical findings at the bottom of this post if you’re interested)

But that’s just me ranting and speculating!

Allow me to talk about something personal for a moment. 95% of the content of OpenRegulatory consists of me ranting about regulations and their terrible implementations (yes, EUDAMED). And the remaining 5% consist of us, as a company, actually fixing these problems, with our eQMS software being the most prominent example of automating regulatory pain away.

Now, as I’m growing old, in my fifth year running OpenRegulatory, I’ve come to realize that the 95% of ranting don’t make me a particularly happy person. So let’s see how we can fix things!

Can we fix EUDAMED?

The answer is yes.

Here’s what we did: We built a better EUDAMED. We call it BEUDAMED.

Here’s what makes it better:

• Searches take milliseconds, not seconds. Just like Google.
• Super simple search interface: Only one text field to search. Filter results later.
• Dynamic filtering: Filter by device type, and even Notified Body, Competent Authority and Authorized Representative!
• Suggestions while you type.
• You can right-click on links to them in new tabs. This is the killer feature.

BEUDAMED is available starting today, for free. Click here to access it!

We believe that sharing information openly and transparently leads to more startups entering the market at lower cost, improving innovation and, ultimately, delivering better Healthcare to patients. That’s why we published our MDR templates for free, and that’s why we built BEUDAMED.

Mario Draghi recently wrote in his recent report that the EU has to increase innovation and, to stay competitive with the US and China, increase productivity. Maybe reducing search times from 18 seconds to less than one second might be a small contribution to that.

We developed this on our own time, with no profit motive in mind.

Enjoy BEUDAMED.

Technical Details

This is the end of the “actual” article. Here are some further technical details. Feel free to skip these unless you’re a technical person and/or interested in them.

EUDAMED API Documentation

Interacting with the EUDAMED database is non-trivial as it doesn’t provide an API. This is in stark contrast to the FDA which has a great initiative called openFDA – it provides great APIs for interacting with FDA data, and it also provides downloadable data dumps for offline processing.

EUDAMED doesn’t have any of that.

But now we’re filling that gap by creating a community-driven, open-source documentation of the EUDAMED API. You can find the EUDAMED API documentation here, and its underlying GitHub repository here. Contributions welcome!

EUDAMED Craziness

I don’t have any insight into EUDAMED’s team and software engineering practices, but what we can see from the outside gives us hints on how terrible it might be:

• Searches often take >10 seconds. This gives us hints that the underlying architecture is not fit for its purpose – maybe they’re not properly indexing their database, or they’re using an unsuitable database (probably should use Elasticsearch or so), etc.
  Imagine Google searches taking more than 10 seconds. People would switch back to AltaVista!
• The frontend seems to be some sort of AngularJS monstrosity. Not the best choice for a framework. Then again, framework choices aren’t necessarily bad if they are used in the right way. See the next point.
• You can’t right-click on links. So it looks like they directly hook into the onClick event in the HTML DOM instead of using “normal” <a href> links. It’s likely that only junior / inexperienced developers would do this.
• The data model is brutally complex. The structured (JSON) data contains so many keys that it takes you multiple days to understand the data (I still haven’t fully understood it). Who’s at fault here? It’s probably some combination of “design by committee”, the UDI situation already being a mess and now being built into the software, and no one having the authority to pound their first on the table and make some executive decisions to simplify things.
• The data has duplication, and it’s inconsistent. Sometimes, similar data is returned in a slightly different way across endpoints (e.g. on manufacturers and authorized representatives). Other times, it’s simply duplicated across keys (e.g. deviceCertificateInfoList and deviceCertificateInfoListForDisplay – both contain the same data – what the hell?). Same here, more hints for junior and/or inexperienced developers; maybe also being under time pressure, as ironic as it sounds, given the lengthy development time so far. Check out our API documentation for details.
• Inconsistent handling of values. E.g. boolean keys not only have true / false as values, but also null.
• Inconsistent naming of keys. Some keys in the data are named differently than in the UI (e.g. contact lenses).

So.. it’s a mess. I don’t think it’s unsalvageable, but I do think that we’ll be seeing more EUDAMED timeline delays as this sort of mess doesn’t really enable fast development of new features on a predictable timeline.

I don’t have any insight into EUDAMED’s team and software engineering practices, but I think it’s fair to say that what we’re seeing from the outside might be glimpses of what sort of chaos might be going on on the inside.

At the very least, EUDAMED can be seen as a typical outcome of “design by committee”: It’s core data structures are so cluttered and convoluted which gives you the feeling that everyone and their dog was allowed to make suggestions, and no one felt responsible for simplifying the overall design. And then, probably, those unpolished suggestions got handed down to a bunch of poor software engineers who had to implement these terrible suggestions without being able to voice their feedback “up the chain” on how truly terrible those ideas were.

Yes, this leads to terrible software. But, having worked with software developers in the past, I’d like to mention another aspect which is even worse: These setups usually burn out software engineers fairly fast, and the best ones tend to leave. I wouldn’t be surprised if EUDAMED had high software developer turnover and is now stuck with a bunch of junior engineers unfit for this sort of large-scale project (no offense), simply because they don’t have other job options.

The EUDAMED Audit?

Here’s the most recently revised EUDAME timeline:

One slightly surprising thing is that EUDAMED is currently being audited (yellow box, “First 5 Modules”).

Many obvious questions here: Why the hell would you do that? Do you expect an audit to lead to better software quality? Etc., etc.

But let’s focus on the duration, because, hold on, it’s being audited for 9 months.

How do you even audit a software for 9 months? I wrote up my technical findings above in two days. I don’t think my list is complete, but fixing those would be useful. Will the 9-month-audit come to similar conclusions? Given that they literally spend ~100x more time than me on this, will their suggestions be 100x more useful?

Talking about productivity..

Team, Development Time & Productivity

It took us one developer and 19 days of development to build BEUDAMED. In contrast, EUDAMED has been under development since ~2017. We don’t even know who is working on it at the EU, so we can only estimate the team size. Let’s assume a team of three developers. That results in around 5,800 person-days spent on EUDAMED development so far.

So 5,800 person-days vs. 19 person-days. That’s a 300x difference in efficiency.

Cool.

Okay okay, yes, this is a bit of an apples-to-oranges comparison: The EUDAMED team had to spend time on the upfront work modelling the data structures (not that the end result is super great), and they also had to build the whole “entering data” kind of stuff which our solution doesn’t have. Sure, the EUDAMED team had more work. But 300x more work?

What even makes a software team 300x slower? The the EU bureaucracy? I’ve read somewhere that they have their own internal framework, called AS4. Browsing the documentation feels like reading the MDR: It’s super lengthy, borderline unreadable and looks like it was written by a non-technical person pretending to be technical. At the very least, this scares away good developers who enjoy writing clean code and shipping features. At worst, it will make progress grind to a halt.

At least EUDAMED gets audited; BEUDAMED does not.

But, then again, BEUDAMED has right-clicking, and EUDAMED does not.