Gutahura ibitagenda neza ni intwari ituje mu mikorere y'amakuru - inzogera y'umwotsi ivugiriza mbere y'uko ibintu bitangira gushya.
Mu magambo asobanutse: Ubuhanga bwo gukora imibonano mpuzabitsina (AI) bumenya uko "ibisanzwe" bisa, bugaha ibintu bishya amanota adasanzwe , hanyuma bugafata icyemezo cyo kureba umuntu (cyangwa guhagarika ikintu mu buryo bwikora) hashingiwe ku mubare runaka . Ishitani iri mu buryo usobanura "ibisanzwe" iyo amakuru yawe ari mu bihe by'umwaka, ari akajagari, agenda ahindagurika, kandi rimwe na rimwe akakubeshya. [1]
Ingingo ushobora gukunda gusoma nyuma y'iyi:
🔗 Impamvu ubuhanga bwo gukora imibonano mpuzabitsina bushobora kwangiza sosiyete.
Isuzuma ingaruka z’imyitwarire, ubukungu, n’imibereho ziterwa no gukoresha uburyo bwo gukora imibonano mpuzabitsina bukwirakwira.
🔗 Umubare w'amazi akoreshwa na sisitemu za AI.
Bisobanura uburyo bwo gukonjesha ikigo cy'amakuru, ibyo amahugurwa akeneye, n'ingaruka z'amazi ku bidukikije.
🔗 Icyo urutonde rw'amakuru rwa AI ari cyo n'impamvu ari ingenzi.
Bisobanura urutonde rw'amakuru, ibirango, amasoko, n'uruhare rwabyo mu mikorere y'icyitegererezo.
🔗 Uburyo ubuhanga bwo gukora ibishushanyo mbonera buhanura ibipimo bivuye mu makuru agoye. Bugaragaza
uburyo bwo kumenya imiterere, imashini zikoresha mu kwigira, n'uburyo bwo guhanura ibintu mu buryo bufatika.
"Ni gute AI imenya ibintu bidasanzwe?"
Igisubizo cyiza kigomba gukora ibirenze algorithme zo ku rutonde gusa. Kigomba gusobanura imiterere n'uko bisa iyo ubishyize mu bikorwa ku makuru nyayo kandi adatunganye. Ibisobanuro byiza:
-
Erekana ibintu by'ibanze: imiterere , imiterere y'ibanze , amanota , n'ibipimo . [1]
-
Huza imiryango ifatika: intera, ubucucike bw'abantu, icyiciro kimwe, kwigunga, gushoboka, kongera kubaka. [1]
-
Gukemura ibibazo by’uruhererekane rw’igihe: “ibisanzwe” biterwa n’igihe cy’umunsi, umunsi w’icyumweru, ibyo wasohoye, n’iminsi mikuru. [1]
-
Fata isuzuma nk'imbogamizi nyayo: ibikangisho bitari byo ntibibabaza gusa - bitwika icyizere. [4]
-
Shyiramo uburyo bwo gusobanura + uburyo umuntu abyumva, kuko "bitangaje" atari impamvu nyamukuru. [5]
Imiterere y'ibanze: Imiterere y'ibanze, amanota, imbibi 🧠
Imiterere myinshi y’ibintu bidasanzwe, byaba byiza cyangwa bitameze neza, igizwe n’ibice bitatu bigenda:
1) Ishusho (izwi nka: icyo icyitegererezo gikora )
Ibimenyetso bidakora neza ntabwo bihagije. Ushobora kuba uhanga ibintu (imibare igenda ihinduka, ikigereranyo, igihe cyo kudindira, delta z'ibihe) cyangwa ukiga ibyo uhagarariye (ibyongera, imyanya mito, kongera kubaka). [1]
2) Gutanga amanota (azwi nka: ibi ni "bitangaje" gute?)
Ibitekerezo bikunze kugaragara byo gutanga amanota birimo:
-
Ishingiye kure : kure y'abaturanyi = biteye amakenga. [1]
-
Ishingiye ku bucucike : ubucucike buke bwo mu gace = biteye amakenga (LOF ni umwana wa posita). [1]
-
Imipaka y'icyiciro kimwe : kwiga "ibisanzwe," kugaragaza ibigwa inyuma. [1]
-
Ibyiza : amahirwe make munsi ya moderi ishyizwemo = biteye amakenga. [1]
-
Ikosa ryo kongera kubaka : niba icyitegererezo cyatojwe ku buryo busanzwe kidashobora kongera kucyubaka, birashoboka ko kiba cyarazimye. [1]
3) Gufata uruzitiro (ruzwi nka: igihe cyo gucuranga inzogera)
Ingano zishobora kuba zidahinduka, zishingiye ku gipimo, kuri buri gice, cyangwa zishingiye ku giciro - ariko zigomba gupimwa hakurikijwe ingengo y'imari isanzwe n'ibiciro biri hasi, ntabwo ari ibintu bihinduka. [4]
Ikintu kimwe gifatika cyane: ibikoresho byo gupima ibintu bidasanzwe/ibishya bya scikit-learn bigaragaza amanota y’ibanze hanyuma bigakoresha umurongo ngenderwaho (akenshi ugenzurwa binyuze mu buryo bwo kwanduzanya) kugira ngo bihindure amanota mu byemezo by’ibanze/bidasanzwe. [2]
Ibisobanuro byihuse birinda ububabare nyuma 🧯
Itandukaniro ribiri rikurinda amakosa adasobanutse neza:
-
Gutahura ibintu bidafite aho bihuriye n'ibintu : amakuru yawe yo guhugura ashobora kuba arimo ibintu bidafite aho bihuriye n'ibintu ...
-
Gutahura udushya : amakuru y’amahugurwa afatwa nk’aho ari meza; urimo usuzuma niba ibyavumbuwe bihuye n’imiterere isanzwe yizwe. [2]
Nanone: gushakisha udushya akenshi bifatwa nk'itsinda ry'icyiciro kimwe - kwerekana ibintu bisanzwe kuko ingero zidasanzwe ziba nke cyangwa zitarasobanuwe. [1]
Amafarashi y'akazi adacungwa uzakoresha mu by'ukuri 🧰
Iyo ibirango ari bike (mu by’ukuri biba ari bike), ibi ni byo bikoresho bigaragara mu miyoboro nyayo:
-
Ishyamba ry’Isolation : ikosa rikomeye mu manza nyinshi zijyanye n’imbonerahamwe, rikoreshwa cyane mu bikorwa kandi rigashyirwa mu bikorwa muri scikit-learn. [2]
-
SVM yo mu cyiciro kimwe : ishobora kugira akamaro ariko irashishikarira guhindura no gutekereza; scikit-learn ivuga ku buryo bweruye ko hakenewe gutunganya neza hyperparameter. [2]
-
Igipimo cya Local Outlier Factor (LOF) : amanota ashingiye ku bucucike bwa kera; ni meza cyane mu gihe "ibisanzwe" atari ikintu gisobanutse neza. [1]
Amakipe ya gotcha avumbura buri cyumweru: LOF yitwara mu buryo butandukanye bitewe niba urimo gukora ubushakashatsi bw’ibintu bidasanzwe ku myitozo ugereranije no kumenya ibintu bishya ku makuru mashya - scikit-learn ndetse isaba ubuhanga = ukuri kugira ngo ibone amanota atagaragara neza. [2]
Ishingiro Rikomeye Rigikora Iyo Amakuru Ari Makenga 🪓
Niba uri mu buryo bwa "dukeneye ikintu kitatwibagirwa", imibare ikomeye ntabwo ihabwa agaciro gahagije.
Amanota ya Z yahinduwe akoresha uburyo bwa median na MAD (median absolute deviation) kugira ngo agabanye ubushobozi bwo kumva agaciro karenze urugero. Igitabo cya NIST cya EDA cyerekana uburyo bwa Z yahinduwe kandi kivuga itegeko rikunze gukoreshwa cyane "rishobora kuba ridasanzwe" ku gaciro karenze urugero ka 3.5 . [3]
Ibi ntabwo bizakemura ikibazo cyose kidasanzwe - ariko akenshi ni umurongo wa mbere ukomeye wo kwirwanaho, cyane cyane ku bipimo by'urusaku no kugenzura hakiri kare. [3]
Ukuri kw'uruhererekane rw'ibihe: "Ibisanzwe" biterwa n'igihe ⏱️📈
Ibitagenda neza mu gihe biragoye kuko aho ibintu biherereye niho ingingo yose iherereye: kwiyongera cyane saa sita bishobora kwitezwe; kwiyongera kumwe saa cyenda za mu gitondo bishobora gusobanura ko hari ikintu kirimo gushya. Bityo, sisitemu nyinshi zikora zigaragaza uburyo ibintu bisanzwe hakoreshejwe ibintu bireba igihe (igihe gito, ikirere gihindagurika, amadirishya azunguruka) n'amanota anyuranyije n'icyitegererezo cyitezwe. [1]
Niba wibuka itegeko rimwe gusa: gabanya umubare w'abakoresha serivisi (isaha/umunsi/akarere/urwego rwa serivisi) mbere yuko utangaza ko kimwe cya kabiri cy'abakoresha serivisi zawe ari "ibidasanzwe." [1]
Isuzuma: Umutego w'Ibibazo Bidasanzwe 🧪
Gupima indwara y’ibicurane akenshi ni "urushinge mu gipangu cy’ibyatsi," ibyo bigatuma isuzuma ridasanzwe:
-
Imirongo ya ROC ishobora kugaragara neza mu buryo butari bwo mu gihe ibyiza bidakunze kugaragara.
-
Ibitekerezo byo kwibuka neza akenshi bitanga amakuru menshi ku bijyanye n’imiterere idahwitse kuko bibanda ku mikorere ku cyiciro cyiza. [4]
-
Mu buryo bw'imikorere, ukeneye kandi ingengo y'imari yo gutanga amakuru : ni amatangazo angahe ku isaha abantu bashobora gutanga amakuru batabanje gucika intege? [4]
Gusuzuma inyuma kuri rolling windows bigufasha kubona uburyo bwa classic failure: "bikora neza cyane… ku ikwirakwizwa ry'ukwezi gushize." [1]
Gusobanura & Impamvu nyamukuru: Garagaza akazi kawe 🪄
Gutanga amakuru nta bisobanuro ni nko kubona ikarita y'amayobera. Ni ingirakamaro, ariko birababaje.
Ibikoresho byo gusobanura bishobora gufasha mu kwerekana ibintu byagize uruhare runini mu manota atari yo, cyangwa mu gutanga ibisobanuro by'imiterere "ni iki cyakenera guhinduka kugira ngo ibi bigaragare nk'ibisanzwe?". cyitwa Interpretable Machine Learning ni ubuyobozi bukomeye kandi bw'ingenzi ku buryo busanzwe (harimo n'imiterere ya SHAP) n'imbogamizi zabwo. [5]
Intego si uguhumuriza abafatanyabikorwa gusa, ahubwo ni ukwihuta mu gupima no kugabanya inshuro zisubiramo.
Gutanga amakuru, Gutembera, no Gutanga ibitekerezo 🚀
Abanyamideli ntibaba mu mashusho. Baba mu miyoboro.
Inkuru isanzwe y’ "ukwezi kwa mbere mu ikorwa": icyuma gipima amakuru ahanini gikoresha amabendera, gitanga akazi kenshi, kandi kibura amakuru… ibi biracyafite akamaro kuko biguhatira gutandukanya "ibibazo by'ubuziranenge bw'amakuru" n' "ibitagenda neza mu bucuruzi."
Mu bikorwa:
-
Genzura uburyo ibintu bigenda bihinduka kandi wongere umenye uko imyitwarire ihinduka. [1]
-
Ibyashyizwe mu nyandiko y'amanota + verisiyo y'icyitegererezo kugira ngo ubashe kongeramo impamvu ikintu cyashyizwe mu mapaji. [5]
-
Fata ibitekerezo by'abantu (amatangazo y'ingirakamaro ugereranije n'amajwi y'urusaku) kugira ngo uhindure imbibi n'ibice uko igihe kigenda. [4]
Inguni y'umutekano: IDS n'isesengura ry'imyitwarire 🛡️
Amatsinda y’umutekano akunze kuvanga ibitekerezo bidasanzwe n’uburyo bwo gutahura bushingiye ku mategeko: ishingiro ry’ “imyitwarire isanzwe y’abakira abantu,” hamwe n’umukono na politiki ku miterere mibi izwi. SP 800-94 (Iya nyuma) ya NIST ikomeje kuba umushinga uvugwa cyane wo gusuzuma no gukumira ibikorwa byo gushakisha no gukumira; inavuga ko umushinga wa “Rev. 1” wo mu 2012 utigeze uba uwa nyuma maze nyuma uvanwa mu kazi. [3]
Ubuhinduzi: koresha ML aho bifasha, ariko ntugatererane amategeko arambiranye - ararambiranye kuko arakora.
Imbonerahamwe yo kugereranya: Uburyo buzwi cyane 📊
| Igikoresho / Uburyo | Ibyiza Kuri | Impamvu bikora (mu bikorwa) |
|---|---|---|
| Amanota ya z akomeye / yahinduwe | Ibipimo byoroshye, ibanze ryihuse | Gutera imbere cyane iyo ukeneye "gutera neza bihagije" kandi ukaba ugabanya inzika z'ibinyoma. [3] |
| Ishyamba ry'Icyaro | Ibiranga imbonerahamwe, ibintu bivanze | Ishyirwa mu bikorwa ry’igihe gito kandi rikoreshwa cyane mu bikorwa. [2] |
| SVM yo mu cyiciro kimwe | Uduce duto "dusanzwe" | Gutahura udushya dushingiye ku mupaka; gutunganya ibintu bifite akamaro kanini. [2] |
| Ikintu Kidasanzwe mu gace | Ibintu bisanzwe bikunze kugaragara | Itandukaniro ry'ubucucike ugereranije n'abaturanyi rigaragaza ibintu bidasanzwe byo mu gace runaka. [1] |
| Ikosa ryo kongera kubaka (urugero, uburyo bwo kwandika amakuru mu buryo bwikora) | Imiterere yo mu ngero nini | Imyitozo isanzwe; amakosa manini yo gusana ibintu ashobora kugaragaza ko hari aho byatandukaniye n’ibindi. [1] |
Kode y'uburiganya: tangira ukoresheje uburyo buhamye + uburyo budasobanutse neza, hanyuma wongereho uburyo bugoye aho wishyura ubukode gusa.
Igitabo Gito cy'Imikino: Kuva kuri Zeru kugeza ku Bumenyesha 🧭
-
Sobanura "ibidasanzwe" mu buryo bw'imikorere (ubutinde, ibyago byo kuriganya, gupfusha CPU, ibyago byo kubika ibintu).
-
Tangira ukoresheje ishingiro (imibare ikomeye cyangwa ibipimo byagabanijwe). [3]
-
Hitamo icyitegererezo kimwe kidakurikiranwa nk'ikizamini cya mbere (Isolation Forest / LOF / One-Class SVM). [2]
-
Shyiraho imipaka ukoresheje ingengo y'imari isanzwe , kandi usuzume ukoresheje uburyo bwo gutekereza ku itangazamakuru niba ibyiza ari bike cyane. [4]
-
Ongeraho ibisobanuro + kwandika kugira ngo buri menyesha rishobore gusubirwamo kandi rishobore gukosorwa. [5]
-
Ikizamini cyo gusubira inyuma, kohereza, kwiga, kongera gukoresha uburyo bwo gusimbuka - guhindagurika ni ibisanzwe. [1]
Ushobora kubikora mu cyumweru kimwe… iyo utekereje ko ibihe byawe bidafatanye na kaseti y'amashanyarazi n'ibyiringiro. 😅
Amagambo ya nyuma - Ndare cyane, sinayasomye🧾
Ubuhanga bwo gukora imibonano mpuzabitsina (AI) bumenya ibitagenda neza binyuze mu kwiga ishusho ifatika y' "ibisanzwe," gupima ibinyuranyo, no kugaragaza ibirenga ku rugero runaka. Sisitemu nziza ntizitsinda kubera ko zigaragara neza, ahubwo zitsinda binyuze mu gupima : imirongo ngenderwaho igabanyijemo ibice, ingengo y'imari y'amakuru, umusaruro ushobora gusobanurwa, hamwe n'uburyo bwo gusubiza ibibazo buhindura inzibutso zivuza urusaku mu kimenyetso cyizewe. [1]
Amareferensi
-
Pimentel n'abandi (2014) - Isuzuma ry'uburyo ibintu bishya byavumbuwe (PDF, Kaminuza ya Oxford) soma byinshi
-
Inyandiko za scikit-learn - Udushya n'Ibintu Bidasanzwe Soma byinshi
-
Igitabo cya elegitoroniki cya NIST/SEMATECH - Gutahura ibintu bidafite aho bihuriye n'abandi soma byinshi na NIST CSRC - SP 800-94 (Isoza): Ubuyobozi bwo Gusuzuma no Gukumira Ubwinjira mu Bitero by'Abaturage (IDPS) soma byinshi
-
Saito & Rehmsmeier (2015) - Igishushanyo mbonera cyo kwibuka neza gitanga amakuru menshi kurusha igishushanyo mbonera cya ROC mu gusuzuma ibyiciro bya binary kuri datasets zitaringaniye (PLOS ONE) soma byinshi
-
Molnar - Kwiga Imashini Bishobora Gusobanurwa (igitabo cya interineti) soma byinshi