Avoiding the data silo anti-pattern
Table of Contents
Open data⌗
If your company has data, a.k.a you don’t work at SleepNumber a.k.a your company exists, there are typically dashboards showing data.
Not only are there dashboards but there are probably emails, excel files, word documents, sticky notes, underwear, also showing data. Unless you’re as all powerful as this person, you won’t be able to reverse engineer a number pasted into a large word document. It’s most likely been pulled from a database (nice) and then transformed in Excel (ermmm) and then adjusted by 10% because Steve doesn’t like the message it’ll send to leadership (yikes) and then added to the word document stored in Sharepoint 2016 (v17-finalfinal.docx).
Suffice it say that data is going to proliferate around an organization, it’s going to get messed up, get changed, whatever, that’s what happens that’s what it gets used for.
But it DOES NOT need to happen at the analytics layer. It does NOT need to happen in your database.
Open Table Formats and Other Decisions⌗
Storing data in an Open Table format so it can be readable by a bunch of different DBMS is a good idea. This is especially good if you work in a company that uses a bunch of different DBMS.
Storing the data that is then used in your visualization software as A TABLE in an OPEN TABLE FORMAT is a good idea. Then people don’t have to scrape the dashboard or copy and paste the query somewhere and execute it. Don’t duplicate data, we like one copy please.
We made a web portal⌗
OMG we made one of those internal company tools which is just a glorified {insert data visualization software here}. If you’re going to do this EXPOSE an API or people will come and SCRAPE your website.
Nobody cares about the website. They just want the data. Make it available. At the very least in some way that doesn’t involve scraping your site.
Be the change⌗
Be the change you want to see in the world, make your data available, think about your customers, and remain flexible. Cheers.