Uburyo bwo kunoza moderi za AI

Uburyo bwo kunoza moderi za AI

Igisubizo kigufi: Kugira ngo urusheho kunoza moderi za AI, hitamo imbogamizi imwe y'ibanze (ubukererwe, ikiguzi, ububiko, ubuziranenge, guhagarara, cyangwa umusaruro), hanyuma ufate umurongo wizewe mbere yo guhindura ikintu icyo ari cyo cyose. Banza ukureho ibibangamira imiyoboro y'amazi, hanyuma ushyiremo inyungu nke nko guhuza ubuziranenge no gukusanya; niba ubuziranenge bugumye, komeza kuri compiler/runtime tooling hanyuma ugabanye ingano ya moderi binyuze mu gupima cyangwa gukwirakwiza mu buryo bwa distillation igihe bibaye ngombwa.

Ibintu by'ingenzi byakunzwe:

Imbogamizi : Hitamo igipimo kimwe cyangwa bibiri by'ingenzi; kunoza imikorere ni ahantu hatandukanye, ntabwo ari ukugera ku buntu.

Igipimo : Urutonde rw'imirimo nyayo hamwe na p50/p95/p99, umusaruro uvamo, ikoreshwa, n'ubwinshi bw'ububiko.

Umuyoboro : Kosora uburyo bwo gushyira amakuru mu byiciro, gushyira amakuru mu byiciro, gutunganya no kuyashyira mu byiciro mbere yo gukora kuri moderi.

Gutanga : Koresha uburyo bwo kubika amakuru, kuyashyira hamwe ku bushake, kuyatunganya mu buryo buhuye n'igihe, kandi ukurikirane neza igihe cyo kuyakoresha.

Ibikoresho byo kurinda : Koresha amabwiriza y'agaciro, ibipimo by'imirimo, no kugenzura neza nyuma ya buri mpinduka mu mikorere.

Uburyo bwo kunoza imiterere ya AI (AI Models Infographic)

🔗 Uburyo bwo gusuzuma neza moderi za AI
Ibipimo by'ingenzi n'ingamba zo gupima moderi mu buryo butagoramye kandi bwizewe.

🔗 Uburyo bwo gupima imikorere ya AI ukoresheje ibipimo nyabyo
Koresha ibipimo ngenderwaho, igihe cyo gutinda, ikiguzi, n'ibimenyetso by'ubwiza kugira ngo ugereranye.

🔗 Uburyo bwo gupima moderi za AI mbere yo gukora.
Imikorere y'igerageza ifatika: gutandukanya amakuru, ibibazo by'imihangayiko, no gukurikirana.

🔗 Uburyo bwo gukoresha ubuhanga bwo gukora ibintu (AI) mu guhanga ibikubiye
mu nyandiko Hindura ibitekerezo mu buryo bwihuse hamwe n'ibitekerezo biteguye neza no gusubiramo ibintu.

1) Icyo "Kunoza" bivuze mu bikorwa (Kuko buri wese abikoresha mu buryo butandukanye) 🧠

Iyo abantu bavuze ngo "noza moderi ya AI," bashobora kuba bashaka kuvuga:

  • Bikore vuba (kugabanya igihe cyo gutinda)

  • Hindura bihendutse (amasaha make ya GPU, gabanya amafaranga ukoresha mu bicu)

  • Bigire bito (uburyo bwo kwibuka, uburyo bwo gushyira mu bikorwa ububiko)

  • Bikore neza kurushaho (kunoza ubuziranenge, kugabanya kubona ibintu nk'aho nta bitekerezo bihari)

  • Bikore bihamye kurushaho (itandukaniro rigabanuka, igabanuka ry'umusaruro)

  • Koroshya gutanga (umusaruro, gukusanya, imikorere iteganyijwe)

Dore ukuri kutumvikana: ntushobora gukoresha ibi byose icyarimwe. Gukoresha neza ni nko gukanda umupira - shyira uruhande rumwe imbere urundi ruhande rugasohoka. Si buri gihe, ariko akenshi bihagije ku buryo wagakwiye guteganya uburyo bwo kugabanya.

Mbere yo gukora ku kintu icyo ari cyo cyose, hitamo inzitizi yawe y'ibanze :

  • Niba ukorera abakoresha imbonankubone, witaye kuri latency ya p95 ( AWS CloudWatch percentiles ) n'imikorere ya tail ( uburyo bwiza bwo "latency ya tail" ) 📉

  • Niba uri kwitoza, witaye ku gukoresha GPU mu buryo bujyanye n'igihe no ku ireme ryayo

  • Niba urimo gukoresha ibikoresho, witaye kuri RAM n'imbaraga 🔋

2) Uburyo bwiza bwo gutunganya moderi za AI bumeze ✅

Uburyo bwiza bwo kunoza ibintu si "gukoresha quantization hanyuma usenge" gusa. Ni sisitemu. Uburyo bwiza bwo gushyiraho ibintu akenshi bugira:

  • Ishingiro wizeye
    Niba udashobora kongera umusaruro wawe ubu, ntushobora kumenya ko hari icyo wateje imbere. Byoroshye… ariko abantu barabyirengagiza. Hanyuma barabyizunguruka.

  • Igipimo cy’ingenzi gisobanutse neza
    "Faster" ntigisobanutse neza. "Gugabanya igihe cyo gutinda kwa p95 kuva kuri 900ms kugera kuri 300ms ku manota amwe y’ubuziranenge" ni intego nyayo.

  • Uburyo bwo kurinda ubwiza bw'umukino.
    Intsinzi iyo ari yo yose mu mikorere ishobora gutuma ubwiza bugabanuka bucece. Ukeneye ibizamini, isuzuma, cyangwa nibura imiterere y'ubwenge.

  • Kumenya ibikoresho.
    Uburyo "bwihuta" kuri GPU imwe bushobora kwinjira ku yindi. CPU ni ubwoko bwazo bwihariye bw'akajagari.

  • Impinduka zisubiramo, ntabwo ari uguhindura ibintu mu buryo butangaje
    Iyo uhinduye ibintu bitanu icyarimwe kandi imikorere ikarushaho kuba myiza, ntumenya impamvu. Ibyo bikaba… bitera ubwoba.

Gutunganya neza bigomba kumera nko gutunganya gitari - gukosora gato, gutega amatwi witonze, gusubiramo 🎸. Niba wumva ari nko gukoresha ibyuma, hari ikintu kitagenda neza.

3) Imbonerahamwe yo kugereranya: Amahitamo akunzwe yo kunoza moderi za AI 📊

Hasi hari imbonerahamwe y'uburyo bwo kugereranya ibintu bisanzwe mu buryo bwo kunoza imikorere y'ibikoresho/uburyo bwo kubikoresha. Oya, ntabwo ari "ubutabera" - ubuzima busanzwe na bwo si uko bumeze.

Igikoresho / Amahitamo Abareba Igiciro Impamvu bikora
PyTorch torch.compile ( inyandiko za PyTorch ) Abantu ba PyTorch Ubuntu Gufata imbonerahamwe + amayeri yo gukusanya bishobora kugabanya amafaranga… rimwe na rimwe biba ari ubumaji ✨
Igihe cyo gukora cya ONNX ( inyandiko za ONNX Runtime ) Amakipe yo gushyiraho abakinnyi Ifite ubuntu Guteza imbere ibitekerezo bikomeye, inkunga yagutse, ni byiza mu gutanga serivisi zisanzwe
TensorRT ( inyandiko za NVIDIA TensorRT ) Gushyira mu bikorwa NVIDIA Ibyiza byishyurwa (bikunze guhuzwa) Guhuza kernel mu buryo bukabije + uburyo ikora neza, byihuse cyane iyo ikanze
DeepSpeed ​​( inyandiko za ZeRO ) Amakipe y'imyitozo Ubuntu Uburyo bwo kongera ubushobozi bwo kwibuka (ZeRO n'ibindi). Bishobora kumera nk'aho ari moteri y'indege
FSDP (PyTorch) (inyandiko za PyTorch FSDP ) Amakipe y'imyitozo Ubuntu Ibipimo/ibipimo bya Shards, bituma moderi nini zidatera ubwoba
ingano ya bitsandbytes ( bitsandbytes ) Abahanga mu bya LLM Ubuntu Uburemere buke, kuzigama ububiko bwinshi - ubwiza buterwa, ariko whew 😬
Gutunganya ( Hinton et al., 2015 ) Amakipe y'ibicuruzwa "Igiciro cy'igihe" Abanyeshuri bato bakura imyitwarire, akenshi inyungu nziza mu gihe kirekire
Gukata ( Inyigisho yo gukata PyTorch ) Ubushakashatsi + umusaruro Ubuntu Bikuraho ibiro bidakomeye. Bikora neza iyo bihujwe no kongera gukora imyitozo
Flash Attention / imbuto zivanze ( urupapuro rwa FlashAttention ) Abahanga mu mikorere Ubuntu Kwita ku bintu byihuse, kwibuka neza. Intsinzi nyayo kuri transfoma
Seriveri y'inyongera ya Triton ( Guhuza bihindagurika ) Ops/infrare Ubuntu Gutanga umusaruro, guhuza, imiyoboro y'ubwoko bwinshi - bisa n'iby'ubucuruzi

Guhindura imiterere y'amakuru adasanzwe: "Igiciro" ntigikwiye kuko uburyo bwo gusohora amakuru bushobora kugutwara impera z'icyumweru ukoresheje uburyo bwo gukemura ibibazo, ari nabyo… igiciro. 😵💫

4) Tangira ukoresheje Ibipimo: Umwirondoro Umeze Nk'aho Ubisobanura 🔍

Niba ukoze ikintu kimwe gusa muri iyi nyandiko yose, kora ibi: pima neza.

Mu igerageza ryanjye bwite, "iterambere rikomeye" ryavuye mu kuvumbura ikintu cyoroshye cyane nk'iki:

  • icyuma gitanga amakuru cyicisha GPU inzara

  • Imbogamizi yo gutunganya CPU mbere y'igihe

  • ingano ntoya z'amatsinda zituma kernel itangira gukora

  • gutinda gutanga tokeni (abakoresha tokeni bashobora kuba abagizi ba nabi bacecetse)

  • gucikamo ibice by'ububiko ( inyandiko zo gutanga ububiko bwa PyTorch CUDA )

  • urwego rumwe rw'ibarura ruganza

Icyo gupima (nibura icyiciro)

  • Ubukererwe (p50, p95, p99) ( SRE ku ijanisha ry'ubukererwe )

  • Uburyo bwo gukoresha (ibimenyetso/isegonda, ubusabe/isegonda)

  • Imikoreshereze ya GPU (kubara + ububiko)

  • VRAM / RAM ifite uburebure bwinshi

  • Igiciro kuri buri tokens ya 1k (cyangwa kuri buri gipimo)

Imitekerereze ifatika yo gusesengura imiterere y'ibishushanyo mbonera

  • Shyira imiterere y'ikintu kimwe ukunda (atari ikibazo cy'igikinisho).

  • Andika byose mu gasanduku gato k'ibihangano.
    Yego biraruhije… ariko bizakurinda kwiyica intege nyuma.

(Niba ushaka igikoresho gifatika cyo gutangira na: PyTorch Profiler ( torch.profiler docs ) na Nsight Systems ( NVIDIA Nsight Systems ) ni byo bikekwaho.)

5) Gukoresha amakuru + amahugurwa: The Quiet Superpower 📦🚀

Abantu bashishikajwe cyane n'imiterere y'icyitegererezo bakibagirwa umuyoboro. Hagati aho umuyoboro utwika bucece kimwe cya kabiri cya GPU.

Intsinzi zoroshye zigaragara vuba

  • Koresha uburyo buvanze bwo gukora (FP16/BF16 aho ihagaze neza) ( PyTorch AMP / torch.amp )
    Akenshi ni vuba, akenshi ni byiza - ariko reba imiterere y'imibare.

  • Gukusanya gradient iyo ingano y'itsinda ari nke ( 🤗 Kwihutisha ubuyobozi )
    Bituma optimization ikomeza kudahinduka nta kwiruka kw'ububiko.

  • Gupima ubunini bw'amajwi ( torch.utils.checkpoint )
    Bihindura uburyo bwo kubara mu buryo bwo kwibuka - bituma ibintu binini bishoboka.

  • Gukoresha tokeni neza ( 🤗 Tokenizers )
    Gukoresha tokenizer bishobora kuba ikibazo gikomeye. Ntabwo ari byiza cyane; ni ingenzi.

  • Gutunganya amakuru
    y'abantu benshi, kubika amakuru mu buryo bwa "pinned memory", gutora amakuru mbere y'igihe - ntabwo ari byiza ariko ni ingirakamaro 😴➡️💪 ( Ubuyobozi bwa PyTorch Performance Tuning Guide )

Gutunganya neza ibipimo

Niba urimo gutunganya moderi nini, uburyo bwa PEFT (nk'ama-adapter ya LoRA) bushobora kugabanya cyane ikiguzi cy'imyitozo mu gihe bukomeza gukomera bitangaje ( 🤗 Transformers PEFT guide , LoRA paper ). Iki ni kimwe mu bihe bya "kuki tutabikoze mbere?".

6) Gutunganya urwego rw'ubwubatsi: Ingano y'iburyo ya Modeli 🧩

Hari igihe uburyo bwiza bwo kunoza ni… kureka gukoresha moderi nini cyane ku buryo idashobora gukoreshwa. Ndabizi, gusuzugura 😄.

Hamagara kuri bimwe mu by'ibanze:

  • Hitamo niba ukeneye ubwenge busesuye, cyangwa inzobere.

  • Komeza idirishya ry'aho ibintu biherereye ribe rinini uko bikenewe, ntabwo ari rinini.

  • Koresha icyitegererezo cyatojwe akazi kari imbere (icyitegererezo cyo gushyira mu byiciro imirimo yo gushyira mu byiciro, n'ibindi).

Ingamba zifatika zo gupima urwego rw'iburyo

  • Hindura ukoreshe icyuma gito ku byifuzo byinshi.
    Hanyuma shyira "ibibazo bikomeye" kuri moderi nini.

  • Koresha uburyo bwo gushyiraho ibyiciro bibiri.
    Imiterere y'imashini yihuta, imashini ikomeye igenzura cyangwa igakosora.
    Ni nko kwandika n'inshuti yawe iguhitamo - birakaza ariko bifite akamaro.

  • Gabanya igihe cyo gusohora.
    Amanota yo gusohora atwara amafaranga n'igihe. Iyo moderi yawe ikora nabi, wishyura amafaranga yo gusohora.

Nabonye amakipe agabanya ikiguzi cyane binyuze mu gushyira mu bikorwa umusaruro muke. Bisa nkaho ari ibintu bito. Birakora.

7) Gutunganya amashusho ya Compiler + Graph: Aho umuvuduko uturuka 🏎️

Uru ni urwego rwo "gutuma mudasobwa ikora ibintu bya mudasobwa by'ubwenge".

Ubuhanga busanzwe:

Mu magambo yumvikana: moderi yawe ishobora kuba yihuta mu mibare, ariko ikora buhoro. Abakora porogaramu bakosora bimwe muri ibyo.

Inyandiko zifatika (zizwi nka inkovu)

  • Uku kunoza ibintu bishobora gutuma habaho impinduka mu miterere y'icyitegererezo.

  • Hari moderi zihuta cyane, izindi ntizigenda neza.

  • Hari igihe ubona umuvuduko n'akaga gateye urujijo - nk'aho gremlin yimuwemo 🧌

Nubwo bimeze gutyo, iyo bigenze neza, ni kimwe mu bintu byiza cyane.

8) Gupima, Gukata, Gutunganya: Bito bitarimo kurira (Byinshi cyane) 🪓📉

Iki ni cyo gice abantu bashaka… kuko bisa nkaho ari ubuntu. Birashobora kubaho, ariko ugomba kubifata nk'ukubagwa.

Ingano (uburemere buke/ibikorwa biri hasi)

  • Ni nziza cyane ku muvuduko w'isuzuma no kwibuka

  • Ingaruka: igabanuka ry'ubuziranenge, cyane cyane ku bice by'inyuma

  • Uburyo bwiza: gusuzuma ku igeragezwa nyaryo, atari ku miterere

Uburyohe busanzwe uzumva kuri:

Gukata (kuraho ibipimo)

  • Ikuraho uburemere cyangwa imiterere "idafite akamaro" ( inyigisho yo gukata PyTorch )

  • Ubusanzwe bisaba kongera guhugurwa kugira ngo wongere kugira ireme

  • Ikora neza kurusha uko abantu babitekereza… iyo ikozwe neza

Gusukura (umunyeshuri yigira ku mwarimu)

Iyi ni yo nkunda cyane mu gihe kirekire. Gukoresha distillation bishobora gutuma habaho moderi ntoya ikora kimwe, kandi akenshi irahamye kuruta gupima cyane ( Distillation the Knowledge in a Neural Network ).

Imvugo idatunganye: gusya ni nko gusuka isupu igoye mu kayunguruzo ugafata… isupu nto. Uko si ko isupu ikora, ariko urabyumva 🍲.

9) Gutanga no Gusobanura: Agace Nyako k'Intambara 🧯

Ushobora "gutunganya" icyitegererezo ariko ugakomeza kukigaragaza nabi. Gutanga serivisi niho gutinda no kwishyura biba ukuri.

Gukorera ni byo bitanga agaciro

  • Guhuza (Batching)
    Binoza umusaruro uva kuri interineti. Ariko byongera igihe cyo gutinda iyo urenze urugero. Biringanize. ( Triton dynamic batching )

  • Gukoresha uburyo bwa Caching
    Prompt Caching na KV-cache reuse bishobora kuba byinshi cyane ku buryo busubirwamo. ( Ibisobanuro bya KV cache )

  • Ibisohoka kuri streaming
    Abakoresha bumva ko byihuta nubwo igihe cyose kingana. Uburyo bwo kubona ibintu ni ingenzi 🙂.

  • Kugabanya amafaranga akoreshwa kuri token ku yindi.
    Hari uduce duto dukora akazi k'inyongera kuri token. Gabanya ayo mafaranga akoreshwa kuri token maze utsindira byinshi.

Itondere igihe cyo gutinda ku mugongo

Impuzandengo yawe ishobora kugaragara neza mu gihe p99 yawe ari ibiza. Ikibabaje ni uko abayikoresha babaho mu mugongo. ( “Umugongo utembera” n'impamvu impuzandengo ibeshya )

10) Gukoresha ibikoresho neza: Huza icyitegererezo n'imashini 🧰🖥️

Gukoresha neza nta bumenyi bw'ibikoresho ni nko gutunganya imodoka y'isiganwa udasuzumye amapine. Yego, urashobora kubikora, ariko ni ubupfapfa gato.

Ibitekerezo bya GPU

  • Ubunini bw'ububiko akenshi ni cyo kintu kigabanya umupaka, ntabwo ari igikoresho cyabugenewe cyo kubara

  • Ingano nini z'amatsinda zishobora gufasha, kugeza igihe zitazafasha

  • Guhuza kernel no kunoza uburyo bwo kwita ku bintu ni byinshi cyane kuri transformers ( FlashAttention: IO-aware exact attention )

Ibitekerezo bya CPU

  • Guhuza imigozi, vektori, n'aho ububiko buherereye bifite akamaro kanini

  • Amafaranga y'ibiciro bya tokenis ashobora kuganza ( 🤗 tokenizers "Zihuta" )

  • Ushobora gukenera ingamba zitandukanye zo gupima ingano kurusha izo kuri GPU

Ibitekerezo bya Edge / mobile

  • Ikimenyetso cy'urwibutso kiba icya mbere mu by'ingenzi

  • Ihindagurika ry'igihe cyo gutinda ni ingenzi kuko ibikoresho… bihindagurika cyane

  • Imiterere mito kandi yihariye ikunze gutsinda imiterere minini rusange

11) Ibikoresho byo kurinda byiza: Ntukigire "umuntu ukunda" amakosa 🧪

Intsinzi yose y'umuvuduko igomba kuzanwa no kugenzura ubuziranenge. Bitabaye ibyo uzishimira, wohereze, hanyuma ukabona ubutumwa nk'ubu bugira buti “kuki umufasha avuga nk'umujura w'umujura?” 🏴☠️

Imirinzi ifatika:

  • Ibitekerezo bya zahabu (urutonde ruhoraho rw'ibitekerezo uhora ugerageza)

  • Ibipimo by'imirimo (ubunyangamugayo, F1, BLEU, icyo ari cyo cyose gihuye)

  • Igenzura ry'aho abantu baherereye (yego, mu by'ukuri)

  • Ibipimo byo gusubira inyuma ("ntibirenze X% byemewe")

Nanone kurikirana uburyo bwo gutsindwa:

  • gushushanya imihindukire

  • impinduka mu myitwarire yo kwanga

  • inshuro zo kubona ibintu mu buryo butunguranye

  • igihe cyo kuzamuka kw'ibiciro mu gisubizo

Guhindura imikorere bishobora guhindura imyitwarire mu buryo butangaje. By’umwihariko. Birakaza. Biteganyijwe, mu gihe urebye.

12) Urutonde: Uburyo bwo kunoza moderi za AI intambwe ku yindi ✅🤖

Niba ushaka urutonde rusobanutse rw'imikorere ya Uburyo bwo Gukora neza Modeli za AI , dore imikorere ikunda gutuma abantu bakomeza kuba bazima:

  1. Sobanura intsinzi
    Hitamo ibipimo 1-2 by'ibanze (ubukererwe, ikiguzi, umusaruro, ubwiza).

  2. Pima imiterere y'ibanze
    y'imirimo nyayo, andika p50/p95, ububiko, ikiguzi. ( PyTorch Profiler )

  3. Kosora imbogamizi zo gukoresha imiyoboro y'amazi.
    Gushyira amakuru mu byiciro, kuyashyira mu byiciro, kuyatunganya mbere, no kuyashyira mu byiciro.

  4. Koresha amanota make yo kubara
    , uburyo bwo gukora neza, gukoresha uburyo bwo gukora neza, no gukora neza.

  5. Gerageza gutunganya imbonerahamwe/uburyo bwo gukora isuzuma ry'igihe
    (graph capture), uburyo bwo gukora isuzuma ry'igihe, uburyo bwo gukora isuzuma ry'igihe (operator fusion). ( torch.compile tutorial , ONNX Runtime docs )

  6. Gabanya ikiguzi cy'icyitegererezo.
    Pima witonze, shyiramo amazi mu kato niba bishoboka, gabanya niba bikenewe.

  7. Gutunganya serivisi zo
    kubika amakuru, guhuza amafaranga, gupima umutwaro, no gukosora igihe cyo gutinda gusohoka.

  8. Emeza ubuziranenge
    Kora ibizamini byo gusubira inyuma hanyuma ugereranye ibyavuyemo ukurikije uko byakabaye.

  9. Gusubiramo
    Impinduka nto, sobanura neza inyandiko, subiramo. Ntibyerekana neza - bigira ingaruka nziza.

Kandi yego, iki ni uburyo bwo kunoza moderi za AI nubwo byaba bisa nkaho ari "Uburyo bwo kureka gukandagira ku mashini." Ni kimwe.

13) Amakosa asanzwe (kugira ngo utayasubiramo nk'uko abandi babikora) 🙃

  • Gutunganya mbere yo gupima
    Uzatakaza umwanya. Hanyuma uzatunganya ikintu kitari cyo neza…

  • Gukurikirana ibipimo ngenderwaho kimwe gusa
    ni ukwirengagiza. Imirimo yawe ni ukuri.

  • Kwirengagiza
    ibibazo byo kwibuka (memory) Bitera gutinda, impanuka, no guhungabana. ( Gusobanukirwa ikoreshwa rya CUDA muri PyTorch )

  • Gupima cyane hakiri kare.
    Umubare muto ushobora kuba mwiza, ariko banza utangire n'intambwe zitekanye.

  • Nta gahunda yo gusubiza inyuma
    Niba udashobora gusubira inyuma vuba, buri gikorwa cyose kiba gihangayikishije. Stress itera udukosa.

Inyandiko Isoza: Uburyo bwa Muntu bwo Kunoza 😌⚡

Uburyo bwo kunoza moderi za AI si ikintu kimwe gusa. Ni inzira igizwe n'ibice bitandukanye: pima, kosora umuyoboro, koresha compilers na runtime, hindura serving, hanyuma ugabanye moderi ukoresheje quantization cyangwa distillation niba ukeneye. Bikore intambwe ku yindi, komeza uburinzi bwiza, kandi ntukizere ko "birushaho kuba byihuta" nk'igipimo (ibyiyumvo byawe ni byiza, ibyiyumvo byawe si porogaramu).

Niba ushaka ikintu kigufi cyane cyo gufata:

  • Pima mbere 🔍

  • Tegura umuyoboro w'amazi ukurikira 🧵

  • Hanyuma ongera uburyo bwo gukoresha icyitegererezo 🧠

  • Hanyuma ongera serivisi nziza 🏗️

  • Komeza kugenzura ubuziranenge buri gihe ✅

Kandi niba bigufasha, ibuka: intego si "icyitegererezo cyiza." Intego ni icyitegererezo cyihuse, gihendutse, kandi cyizewe bihagije ku buryo ushobora gusinzira nijoro… amajoro menshi 😴.

Ibibazo Bikunze Kubazwa

Icyo kunoza icyitegererezo cya AI bivuze mu bikorwa

"Kunoza" ubusanzwe bivuze kunoza imbogamizi imwe y'ingenzi: gutinda, ikiguzi, kwibuka, gukoresha neza, gutuza, cyangwa gutanga umusaruro. Igice kigoye ni uguhinduranya - gusunika igice kimwe bishobora kugabanya ikindi. Uburyo bufatika ni uguhitamo intego isobanutse (nk'igihe cya p95 cyangwa igihe kijyanye n'ubwiza) no kuyinoza. Udafite intego, biroroshye "kunoza" ariko ugakomeza gutsindwa.

Uburyo bwo kunoza moderi za AI nta kwangiza ireme ryihishe

Fata umuvuduko wose cyangwa impinduka z'ibiciro nk'impinduka zitunguranye. Koresha inzira zo kurinda nko kubara, ibipimo by'imirimo, no kugenzura byihuse aho umuntu aherereye. Shyiraho umurongo usobanutse w'ubwiza bw'ibintu kandi ugereranye ibyavuye mu bushakashatsi ku ruhande rumwe. Ibi bituma "birushaho kwihuta" bitahinduka "kuki byahindutse bidasanzwe mu buryo butunguranye?" nyuma yo kohereza.

Ibyo ugomba gupima mbere yo gutangira gukoresha neza

Tangira ukoresheje percentile za latency (p50, p95, p99), imikorere (token/sec cyangwa requests/sec), ikoreshwa rya GPU, na peak VRAM/RAM. Kurikirana ikiguzi kuri buri inference cyangwa kuri 1k token niba ikiguzi ari imbogamizi. Shyira imiterere nyayo utanga, ntabwo ari ikibazo cy'igikinisho. Kubika "perf journal" bigufasha kwirinda gukekeranya no gusubiramo amakosa.

Intsinzi zihuse kandi zifite ingaruka nke ku myitozo

Gukoresha uburyo bwo guhuza (FP16/BF16) akenshi ni bwo buryo bwa mbere bwihuta cyane, ariko reba ibintu bidasanzwe mu mibare. Niba ingano y'itsinda ari nto, kwegeranya ubuziranenge bw'amajwi bishobora gutuma ibintu bihora neza hatabayeho gukurura ububiko bw'amajwi. Gukoresha uburyo bwo kugenzura amajwi (Gradient checkpointing) bigurana uburyo bwo kubara bw'amajwi menshi kugira ngo habeho ububiko buto, bigatuma habaho imiterere minini. Ntukirengagize uburyo bwo gushyira amashusho mu bikorwa (tokenization) no guhindura uburyo bwo gushyira amakuru mu buryo butunguranye - bishobora gutuma GPU yicwa n'inzara bucece.

Igihe cyo gukoresha torch.compile, ONNX Runtime, cyangwa TensorRT

Ibi bikoresho bibanda ku bijyanye n'imikorere: gufata graph, guhuza kernel, no kunoza graph ya runtime. Bishobora gutanga umuvuduko usobanutse, ariko ibisubizo biratandukanye bitewe n'imiterere y'icyitegererezo n'ibikoresho. Hari uburyo bwo gushyiraho ibintu bisa n'ubumaji; ubundi bugenda gahoro. Tega ko habaho impinduka mu miterere y'ishusho ndetse n'udukosa twa "gremlin" rimwe na rimwe - pima mbere na nyuma y'akazi kawe.

Niba gupima bifite akamaro, n'uburyo bwo kwirinda kujya kure cyane

Gupima bishobora kugabanya kwibuka no kwihutisha icyerekezo, cyane cyane kuri INT8, ariko ubwiza bushobora kugabanuka ku ngero z'inyuma. Amahitamo ya biti nto (nka INT4/k-bit) azana kuzigama kwinshi hamwe n'ibyago byinshi. Akamenyero keza ni ugusuzuma ku igerageza nyaryo no kugereranya umusaruro, ntabwo ari ugutekereza mu nda. Tangira n'intambwe zitekanye mbere na mbere, hanyuma ugende ku rugero rwo hasi gusa niba bikenewe.

Itandukaniro riri hagati yo gukata no gusya kugira ngo hagabanuke ingano y'icyitegererezo

Gukata bikuraho "ibipimo by'uburemere bukabije" kandi akenshi bisaba kongera gutozwa kugira ngo bigarure ireme, cyane cyane iyo bikozwe mu buryo buhanitse. Gukata bitoza abanyeshuri bato kwigana imyitwarire y'umwarimu mukuru, kandi bishobora kuba inyungu ikomeye mu gihe kirekire kuruta gupima cyane. Niba ushaka icyitegererezo gito kimeze kimwe kandi kigakomeza kuba gihamye, gukata akenshi ni yo nzira isukuye.

Uburyo bwo kugabanya ikiguzi cyo gupima no gutinda binyuze mu kunoza serivisi

Gutanga serivisi niho uburyo bwo kunoza ibintu bugaragara: gukusanya amakuru byongera umusaruro ariko bishobora kwangiza igihe cyo gutinda iyo bikozwe cyane, bityo ubikore witonze. Gufata amakuru (gufata amakuru vuba na KV-cache reuse) bishobora kuba byinshi iyo contexts zisubiramo. Gusohora amakuru kuri streaming byongera umuvuduko ugaragara nubwo igihe cyose cyaba gisa. Nanone shakisha hejuru ya token-by-token muri stack yawe - akazi gato ka token-token karahita gatera imbere.

Impamvu gutinda ku mugongo ari ingenzi cyane mu kunoza moderi za AI

Impuzandengo ishobora kugaragara neza mu gihe p99 ari ibiza, kandi abakoresha bakunda kubaho mu mugongo. Ubusanzwe gutinda k'umugongo bituruka ku guhungabana: gucikamo ibice by'ububiko, kwiyongera kwa CPU mbere yo gutunganya, kugabanuka kwa tokeni, cyangwa imyitwarire mibi yo gukusanya amakuru. Niyo mpamvu iyi nyandiko ishimangira ibyiciro by'amapasenti n'imirimo nyayo. Niba ukoresheje p50 gusa, ushobora kohereza uburambe "butunguranye bukumva buke."

Amareferensi

  1. Serivisi za interineti za Amazon (AWS) - AWS CloudWatch percentile (ibisobanuro by'imibare) - docs.aws.amazon.com

  2. Google - Umurizo ku gipimo (uburyo bwiza bwo gutinda ku murizo) - sre.google

  3. Google - Intego z'urwego rwa serivisi (Igitabo cya SRE) - ijanisha ry'igihe cyo gutinda - sre.google

  4. PyTorch - torch.compile - docs.pytorch.org

  5. PyTorch - FullyShardedDataParallel (FSDP) - docs.pytorch.org

  6. PyTorch - Umwirondoro wa PyTorch - docs.pytorch.org

  7. PyTorch - Imvugo za CUDA: gucunga ububiko (inyandiko za CUDA zo gutanga ububiko) - docs.pytorch.org

  8. PyTorch - Uburyo bwo gukora imashini zivanze mu buryo bwikora (torch.amp / AMP) - docs.pytorch.org

  9. PyTorch - torch.utils.checkpoint - docs.pytorch.org

  10. PyTorch - Inyigisho zo Gutunganya Imikorere - docs.pytorch.org

  11. PyTorch - Inyigisho yo gukata - docs.pytorch.org

  12. PyTorch - Gusobanukirwa ikoreshwa rya CUDA mu bubiko muri PyTorch - docs.pytorch.org

  13. PyTorch - torch.compile inyigisho / incamake - docs.pytorch.org

  14. Igihe cyo gukora cya ONNX - Inyandiko z'igihe cyo gukora cya ONNX - onnxruntime.ai

  15. za NVIDIA - TensorRT - docs.nvidia.com

  16. NVIDIA - TensorRT bwapimwe - docs.nvidia.com

  17. NVIDIA - Sisitemu za Nsight - developer.nvidia.com

  18. NVIDIA - Seriveri y'inyongera ya Triton - uburyo bwo guhuza ibintu bukoresha ikoranabuhanga - docs.nvidia.com

  19. za DeepSpeed ​​- ZeRO Icyiciro cya 3 - deepspeed.readthedocs.io

  20. bitsandbytes (bitsandbytes-fondation) - bitsandbytes - github.com

  21. Guhoberana mu maso - Kwihutisha: Inyigisho yo Gukusanya Gradient - huggingface.co

  22. Guhoberana mu maso - Inyandiko za Tokenizers - huggingface.co

  23. Guhoberana mu maso - Transformers: Ubuyobozi bwa PEFT - huggingface.co

  24. Guhoberana mu maso - Transformers: Ibisobanuro bya KV cache - huggingface.co

  25. Guhoberana mu maso - Transformers: "Ibyihuta" by'amatoni (amasomo ya tokenizer) - huggingface.co

  26. arXiv - Gukwirakwiza ubumenyi mu miyoboro y'imitsi (Hinton et al., 2015) - arxiv.org

  27. arXiv - LoRA: Guhindura Inyuguti z'indimi nini mu buryo bwo hasi - arxiv.org

  28. arXiv - FlashAttention: Kwita ku bintu byihuse kandi binoze mu kwibuka neza hamwe no kumenya neza ibintu - arxiv.org

Shaka ubuhanga bwa AI bugezweho mu iduka ryemewe rya AI Assistant Store

Ku bijyanye natwe

Garuka kuri blog