JOMusic@lemmy.ml to World News@lemmy.worldEnglish · 1 month agoOpen-source Deepseek R1 dethrones commercial AI, now allegedly being hit by cyberattackwww.cnbc.comexternal-linkmessage-square50fedilinkarrow-up1198arrow-down19cross-posted to: [email protected]
arrow-up1189arrow-down1external-linkOpen-source Deepseek R1 dethrones commercial AI, now allegedly being hit by cyberattackwww.cnbc.comJOMusic@lemmy.ml to World News@lemmy.worldEnglish · 1 month agomessage-square50fedilinkcross-posted to: [email protected]
minus-squarePieisawesome@lemmy.worldlinkfedilinkEnglisharrow-up4·1 month agoIt’s because LLMs don’t work with letters. They work with tokens that are converted to vectors. They literally don’t see the word “strawberry” in order to count the letters. Splitting the letter probably separates them into individual tokens
It’s because LLMs don’t work with letters. They work with tokens that are converted to vectors.
They literally don’t see the word “strawberry” in order to count the letters.
Splitting the letter probably separates them into individual tokens