Oh, it’s a user space (FUSE) driver. I was rather hoping it was an out-of-tree Linux kernel driver, since using FUSE will: 1) always pass back to userspace, which costs performance, and 2) destroys any possibility of DMA-enabled memory operations (DPDK is a possible exception). I suppose if the only objective was to store files in VRAM, this does technically meet that, but it’s leaving quite a lot on the table, IMO.
If this were a kernel module, the filesystem performance would presumably improve, limited by how the VRAM is exposed by OpenCL (ie very fast if it’s just all mapped into PCIe). And if it was basically offering VRAM as PCIe memory, then this potentially means the VRAM can be used for certain RAM niche cases, like hugepages: some applications need large quantities of memory, plus a guarantee that it won’t be evicted from RAM, and whose physical addresses can be resolved from userspace (eg DPDK, high-performance compute). If such a driver could offer special hugepages which are backed by VRAM, then those application could benefit.
And at that point, on systems where the PCIe address space is unified with the system address space (eg x86), then it’s entirely plausible to use VRAM as if it were hot-insertable memory, because both RAM and VRAM would occupy known regions within the system memory address space, and the existing MMU would control which processes can access what parts of PCIe-mapped-VRAM.
Is it worth re-engineering the Linux kernel memory subsystem to support RAM over PCIe? Uh, who knows. Though I’ve always like the thought of DDR on PCIe cards. All technologies are doomed to reinvent PCIe, I think, said someone from Level1Techs.
In the unlikely case that you have very little RAM and a surplus of video RAM, you can use the latter as swap.
givesomefucks@lemmy.world
on 28 Nov 00:35
nextcollapse
Prebuilts have become about the only way to get a deal for a while now.
possiblylinux127@lemmy.zip
on 28 Nov 03:09
collapse
For now…
jeena@piefed.jeena.net
on 28 Nov 01:06
nextcollapse
This is very unfortunate, about a year ago I built my PC and only put in 32 GB of Ram, It was double I had on my laptop so I thought it should be enough for the beginning and I could buy more later.
Already after 2 months I realizes I can do so much more because of the fast CPU in parallel but suddenly the amount of RAM became the bottleneck. When I looked at the RAM prices it didn’t seem quite worth it and I waited. But that backfired because since then the prices never went down, only up.
What are you running that needs more than 32Gb? I’m only just barely being bottlenecked by my 24Gb when running games at 4k
hoshikarakitaridia@lemmy.world
on 28 Nov 02:37
nextcollapse
AI or servers probably. I have 40gb and that’s what I would need more ram for.
I’m still salty because I had the idea of going cpu & ram sticks for AI inference literally days before the big AI companies. And my stupid ass didn’t buy them in time before the prices skyrocketed. Fuck me I guess.
NotMyOldRedditName@lemmy.world
on 28 Nov 03:00
nextcollapse
It does work, but it’s not really fast. I upgraded to 96gb ddr4 from 32gb a year or so ago, and being able to play with the bigger models was fun, but it’s not something I could do anything productive with it was so slow.
You can have applications where wall clock tine time is not all that critical but large model size is valuable, or where a model is very sparse, so does little computation relative to the size of the model, but for the major applications, like today’s generative AI chatbots, I think that that’s correct.
NotMyOldRedditName@lemmy.world
on 28 Nov 03:12
collapse
Ya, that’s fair. If I was doing something I didn’t care about time on, it did work. And we weren’t talking hours, it it could be many minutes though.
possiblylinux127@lemmy.zip
on 28 Nov 03:09
collapse
Your bottle necked by memory bandwidth
You need ddr5 with lots of memory channels for it to he useful
hoshikarakitaridia@lemmy.world
on 28 Nov 08:49
collapse
I always thought using ddr5 average speeds with like 64gb in sticks on consumer boards is passable. Not great, but passable.
Yep, used ECC server RAM DDR3 or DDR4 is basically thrown out. Unfortunately most consumer mainboards do not support ECC.
hoshikarakitaridia@lemmy.world
on 29 Nov 15:09
collapse
This is exactly the reason I’m about to order a dell poweredge r630 with Intel xeon 2680 v4 from alibaba.
Also I’ve never ordered from alibaba before so we’ll see if I get scammed xd
jeena@piefed.jeena.net
on 28 Nov 03:22
nextcollapse
Two browsers full of tabs but that is not a problem, but once I start compiling AOSP (which I sometimes want to do for work at home instead in the cloud because it’s easier and faster to debugg) then it eats up all the RAM imediatelly and I have to give it 40 more GB or swap and then this swapping is the bottleneck. Once that is running the computer can’t really do anything else, even the browser struggles.
usernamesAreTricky@lemmy.ml
on 28 Nov 06:51
collapse
Have you tried just compiling it with fewer threads? Would almost certainly reduce the RAM usage, and might even make the compile go faster if it you’re needing to swap that heavily
Last I looked, a few days ago on Google Shopping, you could still find some retailers that had stock of DDR5 (I was looking at 2x16GB, and you may want more than that) and hadn’t jacked their prices up, but if you’re going to buy, I would not wait longer, because if they haven’t been cleaned out by now, I expect that they will be soon.
roofuskit@lemmy.world
on 28 Nov 03:23
nextcollapse
Switch to Linux, double your available RAM for free.
I just had a look, 2nd of April I payed 67,000 KRW for one 16 GB stick, now the same one (XPG DDR5 PC5-48000 CL30 LANCER BLADE White), they only sell them in pairs, a pair costs 470,000 KRW in the same shop, so 235,000 KRW per 16 GB stick. That is a price increase of 250%, god damn.
roofuskit@lemmy.world
on 28 Nov 03:22
nextcollapse
Guess what, GPU prices are about to go back up because they rely on RAM.
… I was thinking about buying a 96GB DDR5 kit from the local computer store a few weeks ago, but wasn’t sure it was actually worth €700. Checked again now and the exact same product costs €1500. I guess that settles it, 32GB will have to be enough for the next couple of years then.
Holytimes@sh.itjust.works
on 28 Nov 14:33
collapse
Iv come to learn over the years. If you want to buy computer parts just do it.
Your actively stupid if you don’t cause some bigger idiot with more money then brains will make a new grift that causes everything to be unaffordable.
Fuck waiting for deals, fuck thinking twice. Just fucking buy it and ignore reality around you cause you will be pissed either way.
Either a deal comes and you fucked yourself, or everything goes to the moon and now you have nothing AND your fucked.
I fully agree it’s worth waiting for the thrill of the deal.
The Chia crash was great. A local dude I found on Facebook marketplace bought a whole array worth of drives and never got around to even opening the packages for them.
I scored a great local deal on a PC last week, but unfortunately the only weak point of it is the 16GB of DDR5. So I guess for a while I’ll just be chilling with equal amounts of GPU and system memory. Luckily I’m not a gamer and rarely limited by the system memory.
Nvidia reportedly no longer supplying VRAM to its GPU board partners in response to memory crunch — rumor claims vendors will only get the die, forced to source memory on their own
If that’s true, I doubt that they’re going to be coming to earth for long.
I think a bubble pop may be closer than people think. Several F500 company CEOs are at least calling it a bubble now, but admittedly still 100% on board the AI hype train.
threaded - newest
Jam it right in my ass!
You mean ‘ram’ it?
Oh I like you you can come over and fuck my wife.
Just buy an extra GPU and figure out how to turn it’s VRAM into regular RAM. 😌
I know you can create a gpu ram disk so you could use it as swap lol.
Ok, I have to know: how is this done, and what do people use it for?
It uses a github repo that hasn’t been updated in 3 years and it’s more of a gimmick but it’s a fun weekend project.
www.youtube.com/watch?v=qyzBRsfQ_UM
github.com/Overv/vramfs
Oh, it’s a user space (FUSE) driver. I was rather hoping it was an out-of-tree Linux kernel driver, since using FUSE will: 1) always pass back to userspace, which costs performance, and 2) destroys any possibility of DMA-enabled memory operations (DPDK is a possible exception). I suppose if the only objective was to store files in VRAM, this does technically meet that, but it’s leaving quite a lot on the table, IMO.
If this were a kernel module, the filesystem performance would presumably improve, limited by how the VRAM is exposed by OpenCL (ie very fast if it’s just all mapped into PCIe). And if it was basically offering VRAM as PCIe memory, then this potentially means the VRAM can be used for certain RAM niche cases, like hugepages: some applications need large quantities of memory, plus a guarantee that it won’t be evicted from RAM, and whose physical addresses can be resolved from userspace (eg DPDK, high-performance compute). If such a driver could offer special hugepages which are backed by VRAM, then those application could benefit.
And at that point, on systems where the PCIe address space is unified with the system address space (eg x86), then it’s entirely plausible to use VRAM as if it were hot-insertable memory, because both RAM and VRAM would occupy known regions within the system memory address space, and the existing MMU would control which processes can access what parts of PCIe-mapped-VRAM.
Is it worth re-engineering the Linux kernel memory subsystem to support RAM over PCIe? Uh, who knows. Though I’ve always like the thought of DDR on PCIe cards. All technologies are doomed to reinvent PCIe, I think, said someone from Level1Techs.
This is such an incredible write up of something I’ve never even considered to exist. Thank you!
I’d love to have things like that in a form of a post at !graybeard@lemmy.cafe
wiki.archlinux.org/title/Swap_on_video_RAM
The article introduction is gold:
Prebuilts have become about the only way to get a deal for a while now.
For now…
This is very unfortunate, about a year ago I built my PC and only put in 32 GB of Ram, It was double I had on my laptop so I thought it should be enough for the beginning and I could buy more later.
Already after 2 months I realizes I can do so much more because of the fast CPU in parallel but suddenly the amount of RAM became the bottleneck. When I looked at the RAM prices it didn’t seem quite worth it and I waited. But that backfired because since then the prices never went down, only up.
What are you running that needs more than 32Gb? I’m only just barely being bottlenecked by my 24Gb when running games at 4k
AI or servers probably. I have 40gb and that’s what I would need more ram for.
I’m still salty because I had the idea of going cpu & ram sticks for AI inference literally days before the big AI companies. And my stupid ass didn’t buy them in time before the prices skyrocketed. Fuck me I guess.
It does work, but it’s not really fast. I upgraded to 96gb ddr4 from 32gb a year or so ago, and being able to play with the bigger models was fun, but it’s not something I could do anything productive with it was so slow.
You can have applications where wall clock tine time is not all that critical but large model size is valuable, or where a model is very sparse, so does little computation relative to the size of the model, but for the major applications, like today’s generative AI chatbots, I think that that’s correct.
Ya, that’s fair. If I was doing something I didn’t care about time on, it did work. And we weren’t talking hours, it it could be many minutes though.
Your bottle necked by memory bandwidth
You need ddr5 with lots of memory channels for it to he useful
I always thought using ddr5 average speeds with like 64gb in sticks on consumer boards is passable. Not great, but passable.
I’m often using 100gb of cram for ai.
Earlier this year I was going to buy a bunch of 1tb ram used servers and I wish I had.
Damn
Yeah used ram is probably where it’s at. Maybe you get them used later on from data centers…
Yep, used ECC server RAM DDR3 or DDR4 is basically thrown out. Unfortunately most consumer mainboards do not support ECC.
This is exactly the reason I’m about to order a dell poweredge r630 with Intel xeon 2680 v4 from alibaba.
Also I’ve never ordered from alibaba before so we’ll see if I get scammed xd
Two browsers full of tabs but that is not a problem, but once I start compiling AOSP (which I sometimes want to do for work at home instead in the cloud because it’s easier and faster to debugg) then it eats up all the RAM imediatelly and I have to give it 40 more GB or swap and then this swapping is the bottleneck. Once that is running the computer can’t really do anything else, even the browser struggles.
Have you tried just compiling it with fewer threads? Would almost certainly reduce the RAM usage, and might even make the compile go faster if it you’re needing to swap that heavily
Yeah that’s what I’m doing but I played for the fast CPU and can’t make it the bottleneck ^^
Chrome probably
Last I looked, a few days ago on Google Shopping, you could still find some retailers that had stock of DDR5 (I was looking at 2x16GB, and you may want more than that) and hadn’t jacked their prices up, but if you’re going to buy, I would not wait longer, because if they haven’t been cleaned out by now, I expect that they will be soon.
Switch to Linux, double your available RAM for free.
I’ve been on Linux since 2002.
Then use a distribution from 2002 it uses less ram.
/s
Gehe, I wonder how well it would work ^^
I just had a look, 2nd of April I payed 67,000 KRW for one 16 GB stick, now the same one (XPG DDR5 PC5-48000 CL30 LANCER BLADE White), they only sell them in pairs, a pair costs 470,000 KRW in the same shop, so 235,000 KRW per 16 GB stick. That is a price increase of 250%, god damn.
Guess what, GPU prices are about to go back up because they rely on RAM.
GPU is a processor, right? It is the gfx card that will increase in price
You are the worst kind of person.
Respectfully disagree
I also don’t think you are the absolute worst.
Not for long, GPU partners will have to source their own VRAM for manufacturing Nvidia now.
I hope this is the beginning of the end for the cuda monopoly. I just want good gpgpu support for numerical code.
… I was thinking about buying a 96GB DDR5 kit from the local computer store a few weeks ago, but wasn’t sure it was actually worth €700. Checked again now and the exact same product costs €1500. I guess that settles it, 32GB will have to be enough for the next couple of years then.
Iv come to learn over the years. If you want to buy computer parts just do it.
Your actively stupid if you don’t cause some bigger idiot with more money then brains will make a new grift that causes everything to be unaffordable.
Fuck waiting for deals, fuck thinking twice. Just fucking buy it and ignore reality around you cause you will be pissed either way.
Either a deal comes and you fucked yourself, or everything goes to the moon and now you have nothing AND your fucked.
Part of it the thrill of the hunt. I’ve caught some great deals over the years stalking marketplace.
Got .iso storage after chia crashed
Got a 3090 after Bitcoin asics took over
Got a 5900x when the X3D parts came out
But I’ve never seen decent RAM for sale, only single sticks or slow kits.
I fully agree it’s worth waiting for the thrill of the deal.
The Chia crash was great. A local dude I found on Facebook marketplace bought a whole array worth of drives and never got around to even opening the packages for them.
I scored a great local deal on a PC last week, but unfortunately the only weak point of it is the 16GB of DDR5. So I guess for a while I’ll just be chilling with equal amounts of GPU and system memory. Luckily I’m not a gamer and rarely limited by the system memory.
lemmy.today/post/42588975
If that’s true, I doubt that they’re going to be coming to earth for long.
Have to refill before being sent to titan.
Serious question: when will ram prices go down?
When AI dies.
We’re unlikely to see the grand price fall in the coming year. Can get a bit cheaper, though.
Gamers Nexus did a piece on this, but short of a crash or bubble pop, it’s not expected to recover any time soon.
I think a bubble pop may be closer than people think. Several F500 company CEOs are at least calling it a bubble now, but admittedly still 100% on board the AI hype train.
2027
That’s not very capitalism.
2027
Never. Same with GPUs. AI destroyed jobs, the gaming market, and customer service
Buying used RAM on marketplace and hoping it isn’t broken. Hoping it was just stolen from a Best Buy. Fingers crossed y’all!
I just recently bought a Samsung 16GB 5600MT/s stick for 45€ and received a 32GB stick instead! Sorry, but I wanted to brag x)
You made out like a bandit!
Did you let them know?
Zip it, snitch!
I now have stitches
So… the models are all trained up and now they need to run them is what I’m reading.
You need lots of vram to train a model.
An llm once trained can be run in much less vram and a lot of ram
Where are gpu prices coming down? 5070ti is what 800? 3090 24gb is 800-900 still. They not coming down anytime soon.
Yup. Bs headline.
Well that settles my idea of entertaining AM5.
I just bought an Arc A310 for <$200. I couldn’t find a set of 16x2 DDR4 in stock for less than that on Black Friday… Wild times for the RAM market.