We repair the record-keeping in which the same customer has been created three times and the same material sits under two codes. With matching rules, a similarity score and a review queue, duplicate records are deduplicated, a golden record is produced and distributed back to the source systems, and a similar-record warning steps in whenever a new record is created.
The same customer having been created three times in an ERP is the most common data problem we meet in the field. The record was created once with a short version of the company name, the second time with the full registered trading name, the third time in capitals with no punctuation. Nobody does this on purpose: the person creating the record typed into the search box, could not find the result and, being in a hurry, created a new record. An apparently harmless act produces serious consequences on the commercial side. The customer's credit limit is split across three records; one appears to have hit its limit while dispatches carry on against another. The ageing report does not show the real debt, the sales representative takes an old customer for a new one, and two different price lists are applied to the same company. At year end even the top ten customer list comes out wrong, because the same company enters that list as three separate lines.
On the product and material side the consequences are even more concrete. When the same raw material is entered under two different codes on two stock records, there is one material in the warehouse but two balances in the system. Planning sees enough stock under one code and does not raise an order; production discovers at the machine that the other code is empty. The material requirements calculation runs on the wrong input, the stock count does not tally, and because the cost is spread across two records, product profitability comes out wrong as well. What is more, the problem grows by itself: e-commerce, the marketplaces, the field sales application, a trade-fair list and the supplier portal each create their own record. A list cleaned by hand today is dirty again within a few months, because the mechanism that dirties the records is still in place. That is why clean-up campaigns are repeated again and again without result.
Master data management (MDM) puts rules around this work. First it is decided which data counts as master data: customer accounts, suppliers, products, materials, locations. Then matching rules are written; company suffixes and punctuation are stripped from trading names, strong fields such as the tax number, the barcode (GTIN) and the address are compared, and fuzzy matching is used to catch spelling differences. Every candidate pair gets a similarity score: those above the upper threshold merge automatically, those in between land in front of a person as a review queue, those below the lower threshold are not touched at all. The single record that comes out of a merge is called the golden record, and which field wins from which system is written down in advance. The final step is distributing that single record back to the source systems and switching on a similar-record warning when a new record is created; otherwise the same contamination starts all over again the next day.
The honest account of this work is as follows: automatic matching is never one hundred per cent, and we do not promise that it will be. Two genuinely separate companies can have very similar trading names, two companies in the same group may be kept apart deliberately, and in some records the tax number was never entered at all. That is why the review queue is a permanent job and must have an owner inside the company; master data projects without an owner quietly stop after a few months and the list gets dirty again. Merging is also carried out cautiously: it is first tried on a copy, and in the live system it is done in batches and in a way that can be undone. Nor does every company need it; if you have a single system and a few hundred records, cleaning them by hand is cheaper. In discovery we measure the duplication first, and we do not recommend an investment without seeing the number and writing down the reasoning.
Companies where the same customer has been created several times with different spellings, splitting the credit limit and the balance across records. In this set-up the ageing report and collections tracking do not show the real position. The problem distorts sales and dispatch decisions before it reaches accounting: how much goes out to whom, and whose limit is full, is decided on incomplete information, and the risk grows unnoticed.
Set-ups where e-commerce, the marketplaces, the dealer portal and the field sales application each create their own customer and product records. As the number of channels grows, duplication stops being something you can clean by hand; the rule has to live in the system, not in a person. Most projects to bring the channels together in a single panel get stuck at exactly this step.
Plants where the same raw material sits on two stock records under two codes. The stock balance, the count, the material requirements calculation and the product cost are all falsified at once. Because the problem lies in the coding scheme rather than in the warehouse, repeating the stock count does not solve it either; the code structure, the mandatory fields and the rule for creating a record have to be redefined.
Businesses about to migrate to a new system, or to bring two companies' customer and product lists under one roof. Carrying a dirty list into a new system means bequeathing the problem to the new one. The right time to deduplicate is before the migration; a clean-up after the migration is both more expensive and far riskier, because it is done on records that have already seen live transactions.
Which records count as master data — customer accounts, suppliers, products, materials and locations — is decided, along with the mandatory fields, the code structure and the naming rules for each of them. Who creates a record, who approves it and who may change which field is written down. This step is an administrative rather than a technical decision; skip it and the question of who sets the rule blows up in the middle of the project and the work stops.
Company suffixes, punctuation, Turkish character and spacing differences in trading names are cleaned up; fields such as the tax number, the barcode, the telephone number and the address are brought into a common format. Both exact matching and fuzzy text matching then run, and the weight of each field is set separately. The rules are written around your own record-keeping; an off-the-shelf matching template does not give the same result in every sector.
Every candidate pair gets a score. Above the upper threshold the merge is automatic; below the lower threshold nothing is done at all; those in between drop into the review queue, where a person decides by seeing the two records side by side together with their transactions. The thresholds are tuned as samples are validated, not fixed at the start of the project. The decisions taken are put on record, and the rules are improved with that feedback.
A single correct record is produced from the merged ones. Which field wins from which system is written down in advance: tax details may come from the ERP, contact details from the CRM, the delivery address from wherever the field team keeps it up to date. The golden record is therefore not an arbitrary choice but a combination with clear, repeatable rules; which system each field came from remains traceable.
Merging is carried out in batches and in a way that can be undone. Closed codes are kept as aliases; old dispatch notes, invoices and orders do not lose their links and can still be found under the old code. Who merged which two records, when and on what grounds stays in the audit trail. An incorrect merge must be reversible; otherwise the team becomes reluctant to decide and the queue swells.
The deduplicated record is written back to the ERP, the CRM, e-commerce and the field application, or those systems read it from a shared service. More importantly, the point of entry is protected: when a new record is being created, similar records are listed instantly and the user is warned before pressing create new; where necessary, a record cannot be created until a mandatory field is filled in. Clean-up thereby stops being a one-off campaign.
How many candidate pairs there are, how many have merged, how many are waiting in the queue, which mandatory field is left empty how often and which channel produces the most duplicates are all tracked on a single dashboard. Data quality thus stops being a gut feeling and becomes something measured. When the deterioration starts again it is seen early, and the cause can be pinned down channel by channel.
We measure on a read-only copy of the data: how many candidate duplicate records there are, which fields are empty, how consistent the coding is, and which channel produces the duplication. Live data is not touched. The output of this step is not a proposal but a photograph; the investment decision is taken by looking at these numbers, and if it is not needed we say that it is not needed.
The master data domains, mandatory fields, matching rules, score thresholds and source precedence are written together with the departments. In the same exercise, the owner of each master data domain inside the organisation is identified and put on record. No domain is left without an owner; if it is not clear who will run the review queue, that domain is kept out of the first scope.
The rules are run against copied data and a sample of the resulting matches is checked by hand. Wrongly matched and missed pairs are examined one by one, and the rules and thresholds are corrected. This round is repeated several times; depending on the number of records it usually takes two to four weeks, and it is the step that costs the most to skip.
Merging is done in the live system in batches; in every batch the rollback path is kept open and the audit trail is written. Records with a balance, an open order or a contract in force do not merge automatically; they go through manual approval. The golden record and source precedence rules are switched on, and the result is confirmed record by record with the department concerned.
The single records are distributed to the source systems, a similar-record warning is added to the new-record screens, and the duplication dashboard is opened. How to run the queue, how to update the thresholds and what to do when a new system is connected are handed over in writing. After that, support continues with rule maintenance, adding new master data domains and extension.
They will not. The codes of the closed records are kept as aliases and the old documents are linked to the single record; when the history is queried, it is found under both the old code and the new one. Merging is done in batches, with an audit trail, and in a way that can be undone. Records with a balance, an open order or a contract in force do not merge automatically; they go through manual approval. We do not start any merge before these safeguards are in place and have been tried on a copy; getting that order wrong is the most common mistake in this work.
No, and we do not promise that it does. There really are two separate companies whose trading names resemble each other; two companies in the same group may be kept apart deliberately; in some records the tax number was never entered. That is why the system works with three zones: certain matches merge automatically, records that are clearly different are left alone, and the grey area in between goes to a person. A set-up that aims to close the grey area entirely will sooner or later produce a wrong merge, and separating two customer accounts that have been merged in error is far more laborious than a duplicate record.
These are different jobs. Integration carries a record from one system to another; it does not decide which two records describe the same company or the same material. Setting up integration on a dirty structure only means duplicate records spread faster and to more systems. Master data management makes the identity decision and produces the single record; integration is the road that carries that decision to the systems. The two work together and are usually built in the same project; we cover the transport side in detail on the API and systems integration page.
It is not. The merging done in the data warehouse applies to the report only; even if two records appear as one row in the warehouse, two customer accounts still sit in the ERP, and dispatch, the credit limit and invoicing carry on running through those two records. Master data management writes the decision back to the operational system, that is, it solves the problem at source rather than in the report. In practice the two are built together: the warehouse measures duplication and makes it visible, the master data layer corrects it and stops it recurring.
If you use a single system, have a few hundred customer accounts and the right to create records rests with a handful of people, building a separate master data layer is unnecessary; cleaning the list once by hand and tightening the entry rules gives the same result far more cheaply. This work makes sense when the number of records and channels grows, when more than one system can create records, or when an ERP migration or a company merger comes onto the agenda. If the measurement in discovery shows it to be unnecessary, we say so plainly and propose a smaller adjustment instead.
A written definition of the master data domains and the mandatory fields; normalisation and matching rules; a matching engine that runs with score thresholds; a review queue screen; golden record and source precedence rules; an audit trail with the ability to roll back; distribution to the source systems and a similar-record warning on the new-record screens; a duplication and missing-field dashboard. Plus a written handover document explaining how to run the queue. The source code, the rule sets and all the data belong to you.
In a 30-minute discovery call we listen to what you need and tell you honestly whether custom development or an off-the-shelf product is the better answer.
We do not build diagnostic imaging software; your existing PACS stays where it is. What we build is the layer that closes the gap between the device worklist, the patient record, the order, the report and sharing: the image is linked to the right patient, pending orders become visible, and the retention and backup status of the archive becomes auditable.
Details →We build student information systems that bring admissions, enrolment, class placement, timetabling, attendance registers, marks, report cards, parent communication and instalment tracking together on a single record. It is designed around your institution's own calendar and your own fee policy; you do not have to fit into the mould of an off-the-shelf package. The source code and the data belong to the institution.
Details →We build laboratory information management systems that run every step on a single record, from the moment a sample is received through to the certificate of analysis: barcoded sample tracking, a method library, instrument connections, specification checks, staged approval and an audit-ready record structure.
Details →We bring the chain of quote, project, time record, milestone claim and invoice together on a single record. In agencies, consultancies, engineering practices and software firms, who spent how long on which job, resource utilisation and project profitability become visible while the work is still running, not once it has finished.
Details →We do not replace your hospital information management system; we build an operations layer that runs alongside it. Appointments and resource utilisation, treatment plan tracking, reminders, consumable and implant stock, the patient journey in health tourism and the management dashboard all come together in this layer. The source code and the data stay with you.
Details →We take the company's paperwork out of folders and personal computers and move it into a single archive. Every document's version, who may see it, which approval it passed through and how long it will be kept are defined in the system; the document you are looking for is found in seconds rather than minutes, and its history can be evidenced in an audit. A DMS manages the document itself; ECM covers the process and the content that flow with it.
Details →
