uburyo bwo gupima imikorere ya AI

Ni gute wapima imikorere ya AI?

Niba wigeze kohereza moderi yagaragaye neza mu ikaye ariko igakorwa nabi, uba uzi ibanga: uburyo bwo gupima imikorere ya AI si kimwe mu bipimo by'ubumaji. Ni uburyo bwo kugenzura bujyanye n'intego nyazo. Ubunyangamugayo ni bwiza. Kwizerwa, umutekano, n'ingaruka z'ubucuruzi ni byiza kurushaho.

Ingingo ushobora gukunda gusoma nyuma y'iyi:

🔗 Uburyo bwo kuvugana n'ubutasi (AI)
Inyigisho zo kuvugana neza na AI kugira ngo ubone ibisubizo byiza buri gihe.

🔗 Ni iki gitera imbaraga mu buhanga bwa AI?
Isobanura uburyo ubutumwa butuma habaho ibisubizo bya AI n'ubwiza bw'umusaruro.

🔗 Gushyira ibirango ku makuru ya AI ni iki?
Incamake y'uburyo bwo gushyira ibirango nyabyo ku makuru yo guhugura.

🔗 Imyitwarire mbwirizamuco ya AI ni iki?
Intangiriro ku mahame mbwirizamuco ayobora iterambere ry’ubukorano bushingiye ku ikoranabuhanga n’ishyirwa mu bikorwa ry’ikoranabuhanga.

Ni iki gituma imikorere myiza ya AI iba myiza? ✅

Verisiyo ngufi: imikorere myiza ya AI bivuze ko sisitemu yawe ari ingirakamaro, yizewe, kandi ishobora gusubirwamo mu bihe bibi kandi bihinduka. Mu buryo bufatika:

  • Ubwiza bw'umurimo - ubona ibisubizo bikwiye kubera impamvu zikwiye.

  • Gupima - amanota y'icyizere ajyanye n'ukuri, bityo ushobora gufata ingamba zigezweho.

  • Ikomeye - irakomera mu gihe cy'ihindagurika ry'ikirere, imipaka, n'impinduka z'urudaca.

  • Umutekano n'ubutabera - birinda imyitwarire mibi, ibogamye, cyangwa itubahirije amategeko.

  • Ingufu - ni umuvuduko uhagije, ihendutse bihagije, kandi ihamye bihagije kugira ngo ikore ku rugero runini.

  • Ingaruka z'ubucuruzi - mu by'ukuri zihindura KPI ukunda.

Niba ushaka ahantu ho gushingira ku buryo bwemewe bwo guhuza ibipimo n'ibyago, Urwego rwa NIST AI Risk Management Framework ni inyenyeri ikomeye yo mu majyaruguru yo gusuzuma sisitemu neza. [1]

 

Gupima Imikorere ya AI

Uburyo bwo gupima imikorere ya AI ku rwego rwo hejuru 🍳

Tekereza mu byiciro bitatu :

  1. Ibipimo by'imirimo - ubuziranenge bw'ubwoko bw'imirimo: gushyira mu byiciro, gusubira inyuma, gushyira mu byiciro, gukora, kugenzura, n'ibindi.

  2. Ibipimo bya sisitemu - gutinda, umusaruro uva mu guhamagara, ikiguzi kuri buri guhamagara, igipimo cyo gutsindwa, inzibutso zo gusimbuka, SLA z'igihe cyo gukora.

  3. Ibipimo by'umusaruro - umusaruro w'ubucuruzi n'umukoresha wifuza mu by'ukuri: guhindura, kubika amakuru, umutekano, isuzuma ry'amakuru rikorwa n'intoki, ingano y'amatike.

Gahunda nziza yo gupima ibipimo ivanga byose uko ari bitatu nkana. Bitabaye ibyo ubona roketi idasohoka mu kirahure.

Ibipimo by'ingenzi hakurikijwe ubwoko bw'ikibazo - n'igihe cyo gukoresha iki 🎯

1) Gushyira mu byiciro

  • Uburyo bwo gusuzuma neza, kwibuka neza, F1 - itsinda ry’abantu batatu bavuka ku munsi umwe. F1 ni uburyo bwo gusuzuma neza no kwibuka neza; bigira akamaro iyo ibyiciro bidafite ubusumbane cyangwa ibiciro bidafite aho bihuriye. [2]

  • ROC-AUC - urutonde rw'ibipimo by'ibyiciro by'ibyiciro; iyo ibyiza bidakunze kugaragara, reba kandi PR-AUC . [2]

  • Uburinganire buringaniye - impuzandengo y'ibuka mu byiciro byose; ni ingirakamaro ku birango bihindagurika. [2]

Igenzura ry'ibiza: gukoresha neza gusa bishobora kuyobya cyane iyo hatabayeho ubusumbane. Niba 99% by'abakoresha ari inyangamugayo, moderi y'ubupfapfa ihora yemewe ibona amanota 99% kandi igatera ikipe yawe y'uburiganya mbere ya saa sita.

2) Gusubira inyuma

  • MAE ku makosa ashobora gusomwa n'abantu; RMSE iyo ushaka guhana amakosa akomeye; R² ku bijyanye n'itandukaniro ryasobanuwe. Hanyuma genzura neza ikwirakwizwa ry'amakuru n'ibice bisigaye. [2]
    (Koresha ibice bifasha abafatanyabikorwa kumva ikosa.)

3) Gushyira mu byiciro, gushaka amakuru, ibyifuzo

  • nDCG - yita ku mwanya n'akamaro k'amanota; ibipimo ngenderwaho mu ireme ry'ishakisha.

  • MRR - yibanda ku buryo ikintu cya mbere gikenewe kigaragara vuba (ni byiza cyane ku mirimo yo "kubona igisubizo kimwe cyiza").
    (Ingero z'ishyirwa mu bikorwa n'ingero zakozwe ziri mu masomero asanzwe y'ibipimo.) [2]

4) Gukora inyandiko no kuyisesengura

  • BLEU na ROUGE - ibipimo bisanzwe bihurirana; bigira akamaro nk'intangiriro.

  • Ibipimo bishingiye ku gushyiramo amakuru (urugero, BERTScore ) akenshi bifitanye isano ryiza n'ubushishozi bw'umuntu; buri gihe bijyana n'ibipimo by'abantu ku bijyanye n'imiterere, ubudahemuka, n'umutekano. [4]

5) Gusubiza ibibazo

  • Kumenya neza (Exact Match) na F1 yo ku rwego rwa token ni ibintu bisanzwe kuri extractive QA; niba ibisubizo bigomba gutanga amasoko, pima kandi grounding (igenzura ry’ibisubizo).

Gupima, icyizere, na lensi ya Brier 🎚️

Amanota y'icyizere niho sisitemu nyinshi ziba ziherereye bucece. Urashaka amahirwe agaragaza ukuri kugira ngo abahanga bashobore gushyiraho imipaka, inzira igana ku bantu, cyangwa ingaruka ku biciro.

  • Imirongo yo gupima - reba amahirwe yahanuwe ugereranije n'inshuro z'ubuhanga.

  • Amanota ya Brier - itegeko rikwiye ryo gutanga amanota kugira ngo habeho ukuri kw'ibishoboka; hasi ni byo byiza kurushaho. Ni ingirakamaro cyane cyane iyo wita ku bwiza bw'ibishoboka, atari ku rutonde gusa. [3]

Icyitonderwa: F1 "imeze nabi gato" ariko ikora neza cyane ishobora cyane triage - kuko abantu bashobora kwizera amanota.

Umutekano, uruhande, n'ubutabera - pima icy'ingenzi 🛡️⚖️

Sisitemu ishobora kuba nziza muri rusange ariko ikabangamira amatsinda runaka. Kurikirana by'amatsinda n'ibipimo by'ubutabera:

  • Uburinganire bw'abaturage - igipimo cyiza kingana mu matsinda yose.

  • Amahirwe angana / Amahirwe angana - ibipimo by'amakosa bingana cyangwa ibipimo by'ukuri-byiza mu matsinda; ibi bikoreshe mu kumenya no gucunga amasezerano, ntabwo ari nk'amakarita ya pass-fail. [5]

Inama y'ingirakamaro: tangira ukoresheje utubati tw'amakuru dutandukanya ibipimo by'ingenzi n'ibintu by'ingenzi, hanyuma wongereho ibipimo byihariye by'ubutabera uko politiki yawe ibisaba. Bisa nkaho ari ibintu bitoroshye, ariko bihendutse kuruta ikintu cyabaye.

LLM na RAG - igitabo cy'ibipimo gikora neza 📚🔍

Gupima sisitemu zo kubyara ni… ibintu bitoroshye. Kora ibi:

  1. Sobanura ibisubizo kuri buri rugero: ukuri, gufasha, kutangiza, gukurikiza imitako, imiterere y’ikirango, ishingiro ry’inyandiko, ireme ry’ibyangwa.

  2. Hindura isuzuma ry'ibanze ukoresheje imiterere ikomeye (urugero, ibikoresho byo gusuzuma muri stack yawe) kandi ubikomeze hamwe n'amakuru yawe.

  3. Ongeraho ibipimo by'imvugo (bishingiye ku gushyiramo amakuru) hamwe n'ibipimo bihurirana (BLEU/ROUGE) kugira ngo umenye neza. [4]

  4. Gushyira igikoresho mu mwanya wacyo muri RAG: igipimo cyo kugarura amakuru, ubuziranenge/kwibuka neza kw'ibiri mu nkuru, uburyo bwo gusubiza n'uburyo bwo gusubiza bihurirana.

  5. Isuzuma ry'abantu rikumvikana - pima uburinganire bw'abapima (urugero, κ ya Cohen cyangwa κ ya Fleiss) kugira ngo ibirango byawe bitaba ari ibirangaza.

Bonus: andika ijanisha rya latency n'ikiguzi cya token cyangwa kubara kuri buri gikorwa. Nta muntu ukunda igisubizo cy'ubusizi kizagera kuwa kabiri utaha.

Imbonerahamwe yo kugereranya - ibikoresho bigufasha gupima imikorere ya AI 🛠️📊

(Yego ni akajagari gato ku bushake - inyandiko nyazo zirasebanya.)

Igikoresho Abakurikiranira hafi cyane Igiciro Impamvu ikora - gufata vuba
ibipimo bya scikit-learn Abakora muri ML Ubuntu Ishyirwa mu bikorwa ry’amategeko agenga ishyirwa mu byiciro, gusubiramo, gushyira mu byiciro; byoroshye kubishyiraho mu bizamini. [2]
Isuzuma rya MLflow / GenAI Abahanga mu bya siyansi, MLOps Ubuntu + kwishyura Imibare ikoreshwa mu buryo buhuriweho, ibipimo byikora, abagenzuzi ba LLM, amanota yihariye; bandika neza ibihangano by'ubukorikori.
Uko bigaragara Amakipe arashaka dashboards vuba OSS + igicu Ibipimo birenga 100, raporo z'ubuziranenge n'impinduka, uburyo bwo gukurikirana - amashusho meza mu gihe gito.
Ibiro n'amahitamo Amashyirahamwe akomeye y’igerageza Icyiciro cy'ubuntu Igereranya ry'impande, amakuru arambuye, abagenzuzi; imbonerahamwe n'ibisobanuro biratunganye.
LangSmith Abakora porogaramu za LLM Yishyuwe Kora urutonde rwa buri ntambwe, uvange isuzuma ry’abantu n’amategeko cyangwa abasuzuma LLM; ni byiza kuri RAG.
TruLens Abakunzi ba LLM bafunguye OSS Gutanga ibitekerezo bifasha mu gupima uburozi, ishingiro, akamaro; guhuza ahantu hose.
Ibitekerezo Bikomeye Imiryango igenzura ubuziranenge bw'amakuru OSS Shyira mu buryo bwemewe ibyo witeze ku makuru - kuko amakuru mabi atuma ibipimo byose bitagenda neza.
Igenzura ryimbitse Gupima no gusuzuma CI/CD kuri ML OSS + igicu Bateri zari zirimo gupima amakuru anyuranyije n'amategeko, ibibazo by'icyitegererezo, no gukurikirana; inzira nziza zo kurinda.

Ibiciro birahinduka - reba inyandiko. Kandi yego, ushobora kubivanga hatabayeho polisi y'ibikoresho.

Ingano, ikiguzi, n'uburyo bwo gufata ibyemezo - isosi y'ibanga 🧪

Ikintu gitangaje ariko ni ukuri: moderi ebyiri zifite ROC-AUC imwe zishobora kugira agaciro gatandukanye cyane bitewe n'umubare w'ibiciro byawe .

Urupapuro rwihuse rwo kubaka:

  • Shyiraho ikiguzi cy'igipimo cy'impimbano kinyuranye n'igipimo cy'impimbano kinyuranye n'amafaranga cyangwa igihe.

  • Kuramo ibipimo by'ibiciro kandi ubare ikiguzi giteganyijwe kuri buri cyemezo cya 1k.

  • Hitamo ntarengwa cy'ikiguzi giteganyijwe , hanyuma ugifunge ukoresheje uburyo bwo kubikurikirana.

Koresha imirongo ya PR mu gihe ibyiza bidakunze kugaragara, imirongo ya ROC mu ishusho rusange, n'imirongo yo gupima mu gihe ibyemezo bishingiye ku byashobokaga. [2][3]

Isanduku nto: icyitegererezo cyo gupima amatike yo gushyigikira gifite F1 nto ariko nziza cyane yo gupima no gusubiza inzira mu buryo bw'intoki nyuma y'uko imikorere ihindutse kuva ku murongo ukomeye ikajya ku murongo w'icyiciro (urugero, "kwikemura byihuse," "isuzuma ry'abantu," "gutera imbere") bihujwe n'imirongo y'amanota yagenzuwe.

Gukurikirana, gushakisha no gutanga amakuru kuri interineti 🚨

Ama-evals adafite interineti ni intangiriro, ntabwo ari iherezo. Mu ikorwa:

  • Kurikirana uburyo bwo guhindagurika kw'ibicuruzwa , uburyo bwo guhindagurika kw'ibisohoka , n'uburyo imikorere yangiritse hakurikijwe ibice.

  • Shyiraho igenzura ry'inzira zo kurinda - igipimo ntarengwa cyo kubona ibintu nk'ibyo, ibipimo by'uburozi, imiterere y'aho umuntu anyura.

  • Ongeraho dashboards za canary kugira ngo ubone igihe cyo gutinda kwa p95, igihe cyo guhagarara, n'igiciro kuri buri busabe.

  • Koresha amasomero yubatswe ku buryo bwihariye kugira ngo wihutishe ibi; atanga uburyo bwo guhindura ibintu, ireme, no gukurikirana ibintu by’ibanze mu buryo butunguranye.

Imvugo ntoya ifite inenge: tekereza ko icyerekezo cyawe ari nk'igitangira gushya - ntuteka rimwe gusa ngo wigendere; uragaburira, ureba, uhumura, ndetse rimwe na rimwe ukongera ugatangira.

Isuzuma ry'abantu ritasenyuka 🍪

Iyo abantu batanze amanota ku musaruro, inzira iba ifite akamaro kurusha uko ubitekereza.

  • Andika imirongo mito irimo ingero z'imitsindo inyuranye n'imitsindo inyuranye n'imitsindo inyuranye.

  • Kora ingero z'abantu ku buryo butagaragara kandi utazi neza igihe cyose bishoboka.

  • Pima amasezerano hagati y’abapima (urugero, κ ya Cohen ku bapima babiri, κ ya Fleiss kuri benshi) hanyuma wongere uvugurure imbonerahamwe niba amasezerano yacitse.

Ibi bituma ibirango byawe by'abantu bitagenda bitewe n'amarangamutima cyangwa ikawa.

Kwibira mu buryo bwimbitse: uburyo bwo gupima imikorere ya AI kuri LLM muri RAG 🧩

  • Ubwiza bwo kugarura - recall@k, precision@k, nDCG; gusesengura amakuru y’ibyavuye mu izahabu. [2]

  • Gusubiza ubudahemuka - igenzura ryo kwerekana no kwemeza, amanota y'ubudahangarwa, isuzuma ry'abahanganye.

  • Kunyurwa n'umukoresha - ibikumwe, kurangiza imirimo, guhindura intera uvuye ku nyandiko zatanzwe.

  • Umutekano - uburozi, kuvunika kwa PII, kubahiriza politiki.

  • Ikiguzi n'ubukererwe - tokens, cache hits, p95 na p99 ubukererwe.

Bihuze n'ibikorwa by'ubucuruzi: niba guhagarara kw'ubutaka kugabanutse munsi y'umurongo, hita ugana ku buryo buhamye cyangwa isuzuma ry'abantu.

Igitabo cyoroshye cyo gutangira uyu munsi 🪄

  1. Sobanura akazi - andika interuro imwe: icyo AI igomba gukora kandi ikorera bande.

  2. Hitamo ibipimo by'imirimo 2–3 - wongeyeho gupima no nibura igice kimwe cy'ubuziranenge. [2][3][5]

  3. Hitamo ibipimo ukoresheje ikiguzi - ntugakeke.

  4. Kora itsinda rito rya eval - ingero 100–500 zanditseho ibimenyetso bigaragaza uruvange rw'umusaruro.

  5. Hindura evals zawe - isuzuma ry'umuyoboro/igenzura muri CI kugira ngo buri mpinduka ikore igenzura rimwe.

  6. Igenzura mu buryo bw'imikorere - drift, latency, ikiguzi, ibimenyetso by'impanuka.

  7. Suzuma ibipimo bya prune buri kwezi nta muntu ukoresha; ongeramo ibisubiza ibibazo nyabyo.

  8. Andika ibyemezo - ikarita y'amanota ikipe yawe isoma koko.

Yego, ni byo koko. Kandi birakora.

Ibintu bisanzwe bikunze kugaragara n'uburyo bwo kubihunga 🕳️🐇

  • Gushyira ibipimo byinshi ku gipimo kimwe - koresha agasanduku k'ibipimo gahuye n'imiterere y'icyemezo. [1][2]

  • Kwirengagiza kugenzura - icyizere nta kugenzura ni ukwiheba gusa. [3]

  • Nta gutandukanya - buri gihe gabanya ukurikije amatsinda y'abakoresha, imiterere y'ahantu, ibikoresho, ururimi. [5]

  • Ibiciro bitazwi neza - niba udatanze ibiciro ku makosa, uzahitamo umubare utari wo.

  • Guhindagurika kw'impinduka z'imitsi y'abantu - gupima ubwumvikane, kuvugurura ingero, kongera guhugura abasuzuma.

  • Nta bikoresho by’umutekano - ongeramo ubutabera, uburozi, n’igenzura rya politiki ubu ngubu, bitarenze igihe. [1][5]

Interuro waje kugamije: uburyo bwo gupima imikorere ya AI - Igihe kirekire cyane, sinagisomye 🧾

  • Tangira ukoresheje ibisubizo bisobanutse neza , hanyuma ushyire hamwe by'imirimo , sisitemu , n'ubucuruzi . [1]

  • Koresha ibipimo bikwiye kuri ako kazi - F1 na ROC-AUC kugira ngo ushyire mu byiciro; nDCG/MRR kugira ngo ushyire mu byiciro; guhuza + ibipimo by'ubusobanuro mu ikorwa (bihujwe n'abantu). [2][4]

  • Hindura amahirwe yawe hanyuma utange igiciro cy'amakosa yawe kugira ngo uhitemo imbibi. [2][3]

  • Ongeraho ry’ubutabera ukoresheje ibice by’amatsinda kandi ucunge neza amasezerano y’impinduka. [5]

  • Kora evals na monitoring automatique kugira ngo ubashe gusubiramo nta bwoba.

Urabizi uko bimeze - pima icy'ingenzi, bitabaye ibyo uzageraho urusheho kunoza ikitagenda neza.

Amareferensi

[1] NIST. Urwego rw'Imicungire y'Ibibazo bya AI (AI RMF). soma byinshi
[2] scikit-learn. Isuzuma ry'icyitegererezo: gupima ireme ry'ibyahanuwe (Ubuyobozi bw'Umukoresha). soma byinshi
[3] scikit-learn. Gupima amahirwe (imirongo ipima, amanota ya Brier). soma byinshi
[4] Papineni et al. (2002). BLEU: Uburyo bwo gusuzuma mu buryo bwikora imashini ihindura. ACL. soma byinshi
[5] Hardt, Igiciro, Srebro (2016). Uburinganire bw'Amahirwe mu Kwiga Gukurikiranwa. NeurIPS. soma byinshi

Shaka ubuhanga bwa AI bugezweho mu iduka ryemewe rya AI Assistant Store

Ku bijyanye natwe

Garuka kuri blog