Will an LLM be able to match the ground truth >85% of the time when performing PII detection by 2024 end? | Manifold

Will an LLM be able to match the ground truth >85% of the time when performing PII detection by 2024 end?

Plus

25

Ṁ4723

Dec 31

84%

chance

1D

1W

1M

ALL

PII - personal identification information

Stuff like people's names, numbers and codes that identify stuff (SSN, phone number, passport etc), places, locations, names of orgs, attributes that can be used to identify a person, etc.

GPT-4 outperforms Presidio, Microsoft's custom built tool for PII detection. GPT-4 matches ground truth ~77.4% of the times, while it misses a single PII element ~13% of the time.

This question is managed and resolved by Manifold.

#️ Technology

Get

1,000

and

3.00

Sort by:

Assume this includes both false positives and false negatives? What's the denominator?

predictedYES

Just a complete side question, what are the legalities or what are the complicating factors in using a GPT against PII? So, it has to be trained on dummy PII, right? How much dummy PII is needed to train that 85% level you are referring to?

@PatrickDelaney I think microsoft tested against their in house system, which does detect PII on real data

Related questions

LLM Hallucination: Will an LLM score >90% on SimpleQA before 2026?

What will be true of OpenAI's best LLM by EOY 2025?

Will LLMs' loss function achieve the level of entropy of human text by the end of 2030?

Will an LLM improve its own ability along some important metric well beyond the best trained LLMs before 2026?

By 2025 end, will it be generally agreed upon that LLM produced text/code > human text/code for training LLMs?

Will the most interesting AI in 2027 be a LLM?

By 2027, will it be generally agreed upon that LLM produced text > human text for training LLMs?

Will any widely used LLM be pre-trained with abstract synthetic data before 2030?

By 2028 will we be able to identify distinct submodules/algorithms within LLMs?

Will a LLM-based AI be used for a law enforcement decision before 2025?

Related questions

LLM Hallucination: Will an LLM score >90% on SimpleQA before 2026?

Will the most interesting AI in 2027 be a LLM?

What will be true of OpenAI's best LLM by EOY 2025?

By 2027, will it be generally agreed upon that LLM produced text > human text for training LLMs?

Will LLMs' loss function achieve the level of entropy of human text by the end of 2030?

Will any widely used LLM be pre-trained with abstract synthetic data before 2030?

Will an LLM improve its own ability along some important metric well beyond the best trained LLMs before 2026?

By 2028 will we be able to identify distinct submodules/algorithms within LLMs?

By 2025 end, will it be generally agreed upon that LLM produced text/code > human text/code for training LLMs?

Will a LLM-based AI be used for a law enforcement decision before 2025?

© Manifold Markets, Inc.•Terms + Mana-only Terms•Privacy•Rules