Showing posts with label deduplication. Show all posts
Showing posts with label deduplication. Show all posts

Tuesday, June 30, 2015

Not a bug, but a feature

Massive errors mar Aadhaar enrolment (Times of India)
The enrolment process for Aadhaar in Odisha is dogged by massive rejection of data due to errors. According to the directorate of census operations here, enrolled biometric data of 40 lakh people stand rejected by the Unique Identification Authority of India (UIDAI), the Aadhaar body, as on June 15.
Some facts:
Odisha is a state in eastern India. The wiki has its population at 43.73 million as of 2014.
1 lakh = 100,000
1 crore = 10,000,000
All numbers not quoted from the article are in more familiar units.


The article goes on to say a lot about the numbers. 31,700,000 out of 38,400,000 people (82%) of the eligible population have been registered successfully.

The 4 million rejected applications are divided as follows.

2 million were rejected because they were submitted by operators who have been barred from submitting applications. UID works by outsourcing enrollment to private operators who are then paid by the government for accepted applications. Operators who have submitted too many error-riddled or fraudulent applications have been banned from the market.

1 million have been rejected for being duplicate applications, as is proper.

That leaves 1 million true "errors," or failed enrollments that are potentially valid and are described as those submitted on behalf of "very old people and children (between five to 10 years), whose finger prints and iris scans were not registered properly." Now, it may turn out that some of these failed enrollments are duplicate applications as well and it will probably turn out that many (if not most) of these people can be enrolled on a second pass where extra care is taken during the enrollment process. Nevertheless describing 1 million failed enrollments out of 32.7 million presumably legitimate applications as "massive errors" seems uncharitable.

Also, UID contains a "Biometric Exception Clause" which allows for creating UID numbers for people whose biometrics cannot be enrolled. As of May 2015, across India, around 618,000 (0.07%) of UID numbers have been issued with biometric exceptions.

Friday, May 8, 2015

India UID: Interesting de-duplication and exception stats

Over 9 crore Aadhaar enrolments rejected by UIDAI (Zee News)

Out of 823.3 million enrollments, 97.3 million (Approx. 12%) have been rejected for reasons of either quality or duplication.

This may seem to be high to some, or low to others. In the big picture, there is (or should be!) a cost-benefit analysis at the beginning of the project that gets at the expense of the process vs. the infallibility of the process. On the first pass, it might make sense to get the highest proportion of good enrollments with the most convenient process, then to engage in a more expensive enrollment process applied only to more difficult enrollments.

It's also important to note that the 97.3 million rejected enrollments contain both duplicate applications, which must be rejected and other applications where clerical error, fraud, or un-enrollable biometrics are the reason for rejection.

Another interesting statistic in the article is that only about 618,000 UID numbers have been issued under the "Biometric Exception Clause" which allows for creating UID numbers for people whose biometrics cannot be enrolled. That comes out to around 0.07%.

What that means is that (depending on the number of people waiting for a biometric exception) using a data set approaching a billion individuals, at least 99.3% of the population of India is biometrically enrollable within the existing UID enrollment process.

Note: The article uses the Indian numbering units crore and lakh.

1 crore = 10,000,000
1 lakh = 100,000


See also: UID applications without biometrics highly likely fraudulent

Friday, April 20, 2012

Ghana Biometric Voter Roll Deduplication: Sorting Fraud from Ignorance

8m voters registered; 3,829 biometric applicants challenged (Daily Guide)
A total of 3,829 had their applications challenged throughout the country as of the end of the second phase of the biometric voters registration exercise.

Currently, the Electoral Commission (EC) has registered eight million voters.
4,000 multiple registration detected at on-going Biometric Voter Registration-EC (Ghana Web)
Mr Akomea explained that, persons who were found to have registered more than once would have their names removed entirely from the register, be arrested and prosecuted to face either a jail sentence or a fine, as the court may please.

“There is an adjudicating committee looking at those cases. What is important is to establish whether they were done deliberately or some thought their photographs were not nice so they decided to have another registration just that they can have better photographs. The outcome of the investigations should determine what should be done.”
Man Arrested For Registering 15 Times (Graphic) 
A 47-year-old man has been arrested by the Odorkor Police in Accra for registering 15 times in the on-going biometric voters registration (BVR) exercise.
Assuming accurate deduplication, four thousand out of 8 million is a very small fraction (one in 2,000).

There's also a big difference in terms of threats to democracy between the person who re-enrolls to get a different picture and the person who enrolls fifteen times under fifteen different names. There needs to be a way to sort one from the other and that's not something biometrics can do. At a certain point biometric technologies always have to pass the baton to managers and/or policies.

Identity management is about people.

Thursday, April 12, 2012

Ghana: Man Jailed for Attempted Double Voter Registration

14 days for attempted double registration (Modern Ghana)
A 22-year-old resident of Agona Inkum in the Central Region has been sentenced to 14 days' imprisonment for attempting to register twice in the ongoing biometric voters registration exercise.

The convict, John Quaye, was also slapped with a fine of GH¢2,400 by the Agona Swedru Circuit Court or in default serve 12 months' imprisonment with hard labour.
2,400.00 GHS = 1,337.05 USD. That's a lot of cedis.

Wednesday, March 14, 2012

Woman Sentenced for Double Voter Registration

Sierra Leone: Woman Gets Two Years for Double Registration (All Africa)
Fatmata Gbla, 60, has been sentenced to six years imprisonment or pay a fine of Le 500,000 for registering twice in the ongoing biometric voter registration.
5,000,000.00 SLL = 1,144.63 USD
Sierra Leone GDP - per capita (PPP): $800 (2011 est.)

Sierra Leone isn't messing around. For countries serious about reducing rampant vote fraud and the corruption it enables, biometrics can certainly help.

Tuesday, March 6, 2012

Biometric Suppliers Accenture-NEC, L1 ID & Mahindra Satyam-Morpho Lock Horns with UIDAI Over Price

Deduplication: Suppliers want Rs 8; UID offers 2.75 (Economic Times)
Biometric vendors de-duplicating Aadhaar numbers have refused to work with Unique ID Authority of India after finishing their contract, next month. The vendors were eliminating duplicate or redundant information in the Aadhaar IDs at the price of Rs 2.75 per number fixed with the agency.
See also:
India: UID Costs Plummet as Accuracy Remains High

UIDAI dismisses doubts over accuracy of unique ID system (IBN Live India) - for a rare mention of the mathematics involved in deduplication.


h/t @Allevate

Thursday, February 2, 2012

Sierra Leone: Woman Arrested for Double Registration

Detecting the double registration attempt was easy (Awoko)
Barely two days after the first person was caught trying to register twice on the new biometric machines, another person (this time a woman) has also fallen foul of the law by attempting to register twice at Tassoh Island, Ward 348, in Constituency 97.
That's where it gets really interesting as making the arrest turned out to be quite difficult.

ID management is about people.

Monday, January 23, 2012

Malaysia: Biometrics to Enhance Electoral Integrity

While India's getting all the headlines, Malaysia is making great progress in using biometrics to enhance electoral integrity.

Malaysia's biometric voting system ready (Asia One)

Creating a clean electoral roll (The Star)

The articles both make a big deal out of something every biometric electoral system must do, ensure against multiple registrations, but it's nice to see deduplication get the attention it deserves.