Iyo abantu benshi bumvise "ubwenge bw'ubukorano," batekereza ku miyoboro y'ubwonko, algorithms nziza, cyangwa se robots zidasanzwe. Ikintu kidakunze kuvugwa ni iki: Ubuhanga bwo kubika amakuru burya ububiko cyane nk'uko bubikora . Kandi si ububiko bw'ibikoresho gusa buba inyuma bucecetse, bukora akazi kadasanzwe ariko gakenewe cyane ko gutanga amakuru bakeneye.
Reka turebe icyatuma ububiko bw'ibintu buba ingenzi cyane kuri AI, uburyo butandukanye n'uburyo bwa kera bwo kubika ibintu, n'impamvu ari kimwe mu bintu by'ingenzi bituma ububiko bw'ibintu burushaho kwaguka no gukora neza.
Ingingo ushobora gukunda gusoma nyuma y'iyi:
🔗 Ni ikoranabuhanga rigomba kuba ririho kugira ngo hakoreshwejwe ubuhanga bunini bwo gukora imibonano mpuzabitsina (AI) mu bucuruzi
Ibigo by'ikoranabuhanga by'ingenzi bikenewe kugira ngo byongere ubushobozi bwo gukora ikoranabuhanga rya siyansi (AI) neza.
🔗 Gucunga amakuru ku bikoresho bya AI ukwiye kureba
Uburyo bwiza bwo gucunga amakuru kugira ngo wongere imikorere ya AI.
🔗 Ingaruka z'ubwenge bw'ubukorano ku ngamba z'ubucuruzi
Uburyo ubuhanga bwo gukora imibonano mpuzabitsina (AI) bugira ingaruka ku ngamba z'ubucuruzi no mu gufata ibyemezo by'igihe kirekire.
Ni iki gituma ububiko bw'ibintu buba ikimenyetso cy'ubuhanga bwo gukora imibonano mpuzabitsina (AI)? 🌟
Igitekerezo gikomeye: kubika ibintu ntabwo bibangamira amadosiye cyangwa imiterere y’amatsinda akomeye. Bigabanya amakuru mo "ibintu," buri kimwe gifite imiterere y’amakuru. Iyo mizingo ishobora kuba ibintu byo ku rwego rwa sisitemu (ingano, ibihe, urwego rw’ububiko) na tag z’ingenzi zigenwa n’umukoresha [1]. Tekereza nka buri dosiye ifite umurundo w’inyandiko zigufata zikubwira neza icyo ari cyo, uko cyaremwe, n’aho gikwiranye n’umurongo wawe.
Ku makipe ya AI, ubwo buryo bworoshye buhindura ibintu:
-
Igipimo kidatera migraine - Ibiyaga by'amakuru bikwirakwira mu matsinda ya petabyte, kandi ububiko bw'ibintu bubikora byoroshye. Byagenewe gukura ku buryo butagira imipaka no kuramba mu buryo bwinshi (Amazon S3 yirata "11 nines" no gukoporora ahantu hatandukanye mu buryo busanzwe) [2].
-
Ubukungu bwa Metadata - Gushakisha byihuse, imashini zisukuye, n'imiyoboro y'ubwenge kuko imiterere igendana na buri kintu [1].
-
mu bicu - Amakuru aza kuri HTTP (S), bivuze ko ushobora gukurura ibintu mu buryo bungana kandi ugakomeza gukwirakwiza imyitozo.
-
Ubushobozi bwo kwihangana - Iyo umaze iminsi witoza, ntushobora gushyira mu kaga impanuka yo kwica uduce duto two mu gihe cya 12. Kubika ibintu birinda ibyo bitewe n’imiterere yabyo [2].
Muri make ni igikapu kitagira iherezo: wenda imbere hari akajagari, ariko byose biracyaboneka iyo ubifashe.
Imbonerahamwe yihuse yo kugereranya ibintu bya AI 🗂️
| Igikoresho / Serivisi | Ibyiza Ku (Abareba) | Ingano y'ibiciro | Impamvu Ikora (Inyandiko ziri mu nkengero) |
|---|---|---|---|
| Amazon S3 | Amakipe y'ibigo + Cloud-first | Wishyure uko ugenda | Iramba cyane, irakomeye mu karere [2] |
| Ububiko bwa Google Cloud | Abahanga mu bya siyansi n'abashakashatsi ba ML | Ibyiciro byoroshye | Guhuza ML bikomeye, bikomoka ku bicu gusa |
| Ububiko bwa Azure Blob | Amaduka akomeye ya Microsoft | Ifite intera (ishyushye/ikonje) | Nta kibazo kirimo amakuru ya Azure + ML tooling |
| MiniIO | Imiterere ifunguye / Imiterere ya DIY | Kwiyandikisha ku buntu/kwikorera | Ihuye na S3, yoroheje, ikoreshwa ahantu hose 🚀 |
| Igicu Gishyushye cya Wasabi | Amashyirahamwe y’ubucuruzi akoresha amafaranga menshi | Igiciro gito kiri hasi $ | Nta mafaranga yo gusohoka cyangwa gusaba API (kuri buri politiki) [3] |
| Ububiko bw'ibintu bya IBM Cloud | Ibigo binini | Biratandukanye | Agasanduku gakuze gafite amahitamo akomeye y'umutekano w'ikigo |
Buri gihe genzura neza ibiciro ukurikije uko ukoresha ibintu mu buryo busanzwe - cyane cyane iyo usohoka, ingano y'ibyo wasabye, n'uburyo bwo kubika ibintu.
Impamvu amahugurwa ya AI akunda kubika ibintu 🧠
Amahugurwa si "amadosiye make." Ni amamiliyoni menshi y'inyandiko zamenetse icyarimwe. Sisitemu z'amadosiye zihuza cyane. Ububiko bw'ibintu bubyirengagiza hamwe n'ahantu hadasa n'ahasukuye API. Buri kintu gifite urufunguzo rwihariye; abakozi basohora amakuru bakayazana icyarimwe. Amakuru yagabanutse + I/O ihuriranye = GPU ziguma zihugiye aho gutegereza.
Inama ivuye mu miferege: shyira uduce dushyushye hafi y’itsinda rya mudasobwa (agace kamwe cyangwa agace kamwe), kandi ubike neza kuri SSD. Niba ukeneye kohereza amakuru kuri GPU, NVIDIA GPUDirect Storage irakwiye kugenzurwa - igabanya uburyo bwo gukurura CPU, igagabanya igihe cyo gutinda, kandi ikongera bandwidth kuri accelerators [4].
Metadata: Imbaraga ikomeye idahabwa agaciro 🪄
Aha niho ububiko bw'ibintu bugaragara mu buryo budasobanutse neza. Mu gihe cyo gushyira ahagaragara, ushobora gushyiraho amakuru yihariye (nka x-amz-meta-… kuri S3). Urugero, amakuru ya vision ashobora gushyira ku mashusho ukoresheje lighting=low cyangwa blur=high . Ibyo bituma imiyoboro iyungurura, iringaniza, cyangwa ishyira ku murongo idakoresheje dosiye mbisi [1].
Hanyuma hariho verisiyo . Amaduka menshi y’ibintu abika verisiyo nyinshi z’ikintu iruhande ku rundi - zinoze neza ku bushakashatsi bushobora kongera gukorwa cyangwa politiki z’imiyoborere zikeneye gusubizwa inyuma [5].
Ikintu vs Kubika Amadosiye vs Kubika Ibintu ⚔️
-
Ububiko bwa Block : Ni bwiza cyane ku bubiko bw'amakuru bw'ibikorwa - bwihuse kandi busobanutse neza - ariko buhenze cyane ku makuru adatunganijwe neza mu rugero rwa petabyte.
-
Ububiko bw'amadosiye : Buzwi, bworoshye gukoresha POSIX, ariko ububiko bw'amadosiye buracyari hasi bitewe n'imitwaro myinshi ijyanye.
-
Ububiko bw'ibintu : Byakozwe kuva hasi kugera hejuru kugira ngo bigerweho mu buryo bungana, mu buryo bungana, no mu buryo bukoresha amakuru [1].
Niba ushaka ikigereranyo kidasobanutse neza: ububiko bw'amadosiye ni akabati ko kubikamo inyandiko, ububiko bw'amadosiye ni dosiye yo kuri mudasobwa, naho ububiko bw'ibintu ni… umwobo utagira iherezo ufite inyandiko zifata ku buryo runaka zituma ikoreshwa.
Imikorere ya Hybrid AI 🔀
Ntabwo buri gihe biba ari ibintu bikorerwa mu bicu gusa. Uruvange rusanzwe rusa nka:
-
Ububiko bw'ibintu buri imbere (MinIO, Dell ECS) ku makuru y'ibanga cyangwa agenzurwa.
-
Ububiko bw'ibintu mu bicu (cloud object storage) ku mirimo myinshi, kugerageza, cyangwa ku bufatanye.
Iri gereranya rireba ikiguzi, kubahiriza amategeko, no gukora neza. Nabonye amakipe ajugunya terabytes mu ijoro rimwe mu ndobo ya S3 kugira ngo atange GPU y'agateganyo - hanyuma ayikoreshe iyo sprint irangiye. Ku ngengo y'imari iciriritse, moderi ya Wasabi yoroshye guteganya/kutagira iherezo [3] yoroshya ubuzima.
Igice Ntawe Urata 😅
Igenzura ry'ukuri: ntabwo ari nta makemwa.
-
Gutinda - Shyira kure cyane imibare n'ububiko maze GPU zawe zigatangira gukurura. GDS irafasha, ariko imiterere iracyari ingenzi [4].
-
Ibiciro bitunguranye - Amafaranga yo gusohoka no gusaba API arushaho kuba menshi ku bantu. Abatanga serivisi bamwe barayareka (Wasabi irabikora; abandi ntibabikora) [3].
-
Akajagari k'amakuru ku rwego rwo hejuru - Ni nde usobanura "ukuri" mu ma tagi na verisiyo? Uzakenera amasezerano, politiki, n'imbaraga zimwe na zimwe z'ubuyobozi [5].
Kubika ibintu ni ingenzi mu miyoboro y'amazi: ni ingenzi, ariko si ingenzi cyane.
Aho yerekeza 🚀
-
Ububiko bw’ubwenge kandi bukoresha ubuhanga bwa AI bushyira ahagaragara amakuru mu buryo bwikora binyuze mu byiciro by’ibibazo nk’ibya SQL [1].
-
Guhuza ibikoresho bya mudasobwa (inzira za DMA, NIC offloads) bityo GPU ntizibe zishonje cyane [4].
-
Ibiciro bisobanutse kandi biteganyijwe (uburyo bworoshye, amafaranga yo gusohoka adahari) [3].
Abantu bavuga ko kubara ari ejo hazaza ha AI. Ariko se koko? Ikibazo ni nko gushyira amakuru mu buryo bwihuse nta kubangamira ingengo y'imari . Niyo mpamvu uruhare rw'ububiko bw'ibintu rukura gusa.
Incamake 📝
Kubika ibintu si ibintu bitangaje, ariko ni ingenzi. Iyo umuntu atabishoboye, abizi neza kandi adashobora kubibika, gutoza abanyamideli bakomeye bisa nkaho ari nko kwiruka marato yambaye sandali.
Yego rwose-GPU ni ingenzi, frameworks ni ingenzi. Ariko niba ushishikajwe no gukoresha ubuhanga bwa AI, ntukirengagize aho amakuru yawe aba . Amahirwe ni uko ububiko bw'ibintu bumaze guhagarika ibikorwa byose.
Amareferensi
[1] AWS S3 – Amakuru y'ibintu - amakuru ya sisitemu n'uburyo bwihariye
https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingMetadata.html
[2] AWS S3 – Amasomo yo kubika - kuramba (“11 nines”) + kwihangana
https://aws.amazon.com/s3/storage-classes/
[3] Wasabi Hot Cloud – Ibiciro - igiciro gihoraho, nta mafaranga yo gusohoka/API
https://wasabi.com/pricing
[4] NVIDIA GPUDirect Storage – Inyandiko - Inzira za DMA zijya kuri GPU
https://docs.nvidia.com/gpudirect-storage/
[5] AWS S3 – Verisiyo - verisiyo nyinshi zo kuyobora/kongera gukoreshwa
https://docs.aws.amazon.com/AmazonS3/latest/userguide/Versioning.html