176: GreyBeards talk recent news at Future of Memory & Storage Conference with Jim Handy, Objective Analysis

Jim Handy, General Director at Objective Analysis and I were at Future of Memory and Storage 2026 Conference in Santa Clara earlier this month and there was a lot of news going around. Jim’s been on our show just about every year to discuss FMS news, And with the recent focus beyond flash, it’s even harder for one person to keep up.

New tech was the main topic discussed at the show but the one underlying thread was the AI supercycle and the switch from AI training to inference. The supercycle and inference seems to be turning the IT industry on it’s head.

HBM and DRAM were highlights of the show because they are in every accelerator/GPU. For whatever reason, there seems to be no price elasticity with respect to accelerators, DRAM or data centers for that matter.

So the worlds memory fabs are moving from relatively low margin DRAM to the higher margin HBM. Starving the rest of the IT world for memory. Jason mentioned that fab HBM capacity is all contracted for out to the end of 2027.

Speaking of which there was a lot of talk about KV (key-value) data offload (taking data out of HBM and storing it elsewhere while it’s unused) and the IO requirements needed to support an effective KV cache offload . One vendor said their current storage solution served 10M IOPs for a single HBF (high bandwidth flash storage) device but the real solution will need 100M 512B IOPs.

It turns out that token and KV cache elements are relatively small and using 4K blocks wastes a lot of space. And of course with 1000s of processor elements accessing the KV cache during token processing, thinking and other AI inference activities, if the KV data elements are going to be anywhere but in HBM they need to be accessed very quickly.

HBF, located on the accelerator board adjacent to HBM, was a hot topic last year but this year one could see some solutions coming to market. The one vendor discussed above talked about their liquid cooled HBF. But there were others with HBF storage of thier own.

On the other hand there were a number of storage vendors talking about taking KV cache out off the accelerator and onto storage on the PCIe bus and using NVMe SSDs. Not HBF but maybe another tier of KV caching. However there was lot of talk about CXL shared memory pooling that could be used as a shared KV cache to also do this. Unclear which tech (CXL memory pooling or NVMe SSDs) will win this race but it just might be a combination of both of them.

One vendor mentioned a new technology stacking DRAM directly ontop of (accelerator) compute modules. This would eliminate latency required to go off chip to an HBM and would use a chiplet interface. Doing so would create an intermediate tier of accelerator memory between compute SRAM and HBM, cheaper than SRAM and with more capacity.

Hyperscalers are sitting on a lot of DDR4 DRAM which they are salvaging and moving to shared memory pools behind CXL switches. Jason has been talking about this for years but it’s becoming a reality.

Jim mentioned that even without the AI supercycle driving data center build out there would likely have been a DRAM and NAND crunch coming anyway. This just happens every few years as fabs over produce driving prices down, which drives demand up and leads to fabs being unable to satisfy demand. Usually component price increases depress demand which allows the fabs enough time to increase production and start the cycle again.

It’s just that the AI supercycle has driven all this like it’s on steroids. And price increases have yet to dampen demand. The hyperscalers don’t seem to care the price they have to pay to power AI inference.

In fact some hyperscalers are going right to the fabs and contracting for all the chips coming off a fab line for a period of time. This seems to be the only way they can guarantee to support the compute needed for AI inference.

For once there wasn’t much talk about increasing capacity or 3D NAND layers at the show. In prior shows SSD capacity and 3D NAND layer counts were always news.

Last year we saw a glimmer of 3D layer increase backlash as the tradeoffs of more layers vs, more capacity per layer were discussed. This year 3D NAND layers and SSD capacities weren’t even a topic of discussion. The NAND and SSD world are moving to high bandwidth and high capacity flash, talking less about the technology behind them and their capacity.

Jim mentioned one reason for the reluctance to discuss 3D layers increasing was that 3D NAND suppliers were running out of space for stairways. Turns out each layer in 3D NAND (and most are over 100 layers today) has to have a separate electronic connection outside of it and this requires building a stairway with one stair per layer. And as you get to 100 to 200 or more layers, the stairways themselves start consuming too much NAND floor space, reducing capacity.

Finally, MRAM was another hot topic at the show. It turns out at current chip technology (2nm) levels, NOR gates out of normal TTL don’t work that well. MRAM seems impervious to this problem and is increasingly being used to replace TTL NOR gates in semiconductors.

MRAM has been a long time vendor at the show but mostly in niche solutions. For example, MRAM seems pretty impervious to cosmic rays so it’s increasingly used in RAD hardened applications such as in space, nuclear reactors, et al. But this is a major new use case that could broaden MRAM technology ecosystem beyond the niche players. Watch for more in this space.

Jim and the team at Objective Analysis have been busy this year churning out articles and in depth reports on NAND. memory, and compute business and recent technology innovations. Check out their website (link above) to learn more.

Jim Handy, General Director Objective Analysis

Jim Handy of Objective Analysis has over 35 years in the electronics industry, including 20 years as a leading semiconductor and SSD industry analyst. Early in his career he held marketing and design positions at leading semiconductor suppliers including Intel, National Semiconductor, and Infineon.

A frequent presenter at trade shows, Mr. Handy is known for his technical depth, accurate forecasts, widespread industry presence and volume of publication.

He has written hundreds of market reports, articles for trade journals, and white papers, and is frequently interviewed and quoted in the electronics trade press and other media. 

He posts blogs at www.TheMemoryGuy.com, and www.TheSSDguy.com

170: FMS25 wrap-up with Jim Handy, Objective Analysis

Jim Handy, General Director at Objective Analysis and I were at FMS25 in Santa Clara last week and there was a lot of news going around. Jim’s been on our show just about every year to discuss FMS news, And with the recent focus beyond flash, it’s even harder for one person to keep up.

Much of the discussion at FMS was on HBM4, new QLC capacity points, UAlink/UCe for chiplets, 100M IOP SSDs, and more. Listen to the podcast to learn more.

Th.ere was not as much on CXL as in past shows. and ditto on increasing layer counts to drive more NAND capacity. A couple of years ago layer counts were all they talked about. And CXL was the major change to hit the data center. Jim’s view (and Jason’s) was that CXL was as a way for hyperscalers to make use of DDR4 DRAM but that need has passed now.

As for layer counts they are still going up but not as fast. And the economics of 3D scaling now have to compete with 2D scaling and “virtual scaling”.

But UAlink and UCe were active topics both of which are used to tie together chiplets in CPUs to build SoCs. SSD vendors are starting to use chiplet architectures to build their massive capacity SSDs and UAlink/UCe would be a way to architect these.

SLC NAND is back to support very high performance SSDs or as a replacement for SCM (storage class memory or Optane). One vendor talked about reaching 100M (random 512B read) IOPS for a single SSD. Current SCL flash can do ~10M IOPS, next gen is speced to do ~30M and the one following would be 100M. One challenge is that current SSDs do 4Kbyte IO and it still takes a msec. or so to erase a page and reading a page isn’t that fast. But the performance is for read only activity.

HBM4 was one topic at the show but the newest wrinkle was HB Flash, or putting SSDs behind HBM to support GPU caching (SSD to HBM to GPU). This would allow more data to be quickly accessed by a GPU.

Jim also mentioned that there’s some interest in narrowing HBM access width, currently 1Kb and increasing to 2Kb with HBM4. This width, and all the pins it requires, limits how many HBM chips one can surround a GPU with. If HBM had a narrower interface more HBM chips could surround a GPU, increasing memory size and perhaps memory bandwidth. HBM4 seems to be going the wrong way but with narrower width HBM, they could easily double the number of HBM chips surrounding a GPU.

They were also showing off a 40 SSD 2U chassis using E.2 form factor SSDs. Pretty impressive and given the capacity on offer a lot of storage per RU.

Speaking of capacity one vendor announced a 246TB QLC SSD, roughly a 1/4PB in a single SSD. With 24 of these per 2U shelf, one could have a >1/10 Exabyte, (>100 PB) in a 40U rack. It looks like no end in sight for SSD capacities. And we aren’t even talking about PLC yet.

At the other end of SSD capacity, it appears that M.2 SSDs were getting hotter on one side (controller side) than the other, throttling performance. So one vender decided to provide heat (liquid cooling) pipes between the two sides to equalize thermal load.

Jim Pappas (lately of Intel) won the lifetime achievement award from FMS. Jim’s accomplishments span a wide swath of storage technology but at the award ceremony he waxed on his work on the USB connector. He said his will stipulates that once he is interned in the ground, they are to take out the casket and spin it around 180 degrees and put it back down again. 🙂

There were quite a number of side topics not directly related to FMS25 on the podcast which were interesting in their own right, but I think i’ll leave it here.

Jim Handy, General Director Objective Analysis

Jim Handy of Objective Analysis has over 35 years in the electronics industry, including 20 years as a leading semiconductor and SSD industry analyst. Early in his career he held marketing and design positions at leading semiconductor suppliers including Intel, National Semiconductor, and Infineon.

A frequent presenter at trade shows, Mr. Handy is known for his technical depth, accurate forecasts, widespread industry presence and volume of publication.

He has written hundreds of market reports, articles for trade journals, and white papers, and is frequently interviewed and quoted in the electronics trade press and other media. 

He posts blogs at www.TheMemoryGuy.com, and www.TheSSDguy.com

155: GreyBeards SDC23 wrap up podcast with Dr. J Metz, Technical Dir. of Systems Design AMD and Chair of SNIA BoD

Dr. J Metz (@drjmetz, blog), Technical Director of Systems Design at AMD and Chair of SNIA BoD, has been on our show before discussing SNIA research directions. We decided this year to add an annual podcast to discuss highlights from their Storage Developers Conference 2023 (SDC23).

Dr, J is working at AMD to help raise their view from a pure components perspective to a systems perspective. On the other hand, at SNIA, we can see them moving out of just storage interface technology into memory (of all things) and real long term, storage archive technologies.

SDC is SNIA’s main annual conference, which brings storage developers together with storage users to discuss all the technologies underpinning storing the data we all care so much about. Listen to the podcast to learn more

SNIA is trying to get their hands around trends impacting the IT industry today. These days, storage, compute and networking are all starting to morph into one another and the boundary lines, always tenuous at best, seem to be disappearing.

Aside from industry standards work that SNIA has always been known for, they are also deeply involved in education. One of their more popular artifacts is the SNIA Dictionary (recently moved online only), which provides definitions for probably over a 1000 storage terms. But SDC also has a lot of tutorials and other educational sessions worthy of time and effort. And all SDC sessions will be available online, at some point. (Update 10/25/23: they are all available now at Sessions | SDC 2023 website)

SNIA also presented at SFD26, while SDC23 was going on. At SFD26, SNIA discussed DNA data storage which is a recent technical affiliate and a new Smart Data Transfer Interface (SDXI), a software defined interface to perform memory to memory DMA.

First up, DNA storage, the DNA team said that they pretty much are able to store and access GB of DNA data storage today, without breaking a sweat and are starting to consider how to scale that up to TB of DNA storage.  We’ve discussed DNA data storage before on GBoS podcasts (see: 108: GreyBeards talk DNA storage... )

The talk at SFD26 was pretty detailed. Turns out the DNA data storage team have to re-invent a lot of standard storage technologies (catalogs/Indexes, metadata, ECC, etc) in order to support a DNA data soup of unstructured data.

For exampe, ECC for DNA segments (snippets) would be needed to correctly store and retrieve DNA data segments, And these segments could potentially be replicated 1000s of times in a DNA storage cell. And all DNA data segments would be tagged with file oriented metadata indicating (segment) address within file, file name or identifier, date created, etc.

As far as what an application for DNA storage would look like, Dr. J mentioned write once and read VERY infrequently. It turns out while making 1000s of copies of DNA data segments is straightforward, inexpensive and trivial, reading it is another matter entirely. And as was discussed at SFD26, reading DNA storage, as presently conceived, is destructive. (So maybe having lots of copies is a good and necessary idea.)

But the DNA guru’s really have to a come up with methods for indexing, searching, and writing/reading data quickly.  Todays disks have file systems that are self-defining. If you hand someone an HDD, it’s fairly straightforward to read information off of it and determine the file system used to create it. These days, with LTO-FS, the same could be said for LTO tape.

DNA is intended to be used to store data for 1000s of years. They have retrieved intact DNA from a number of organisms that are over 50K years old.  Retaining applications that can access, format and process data after a 1000 years is yet another serious problem someone will need to solve.

Next up was SDXI, a software defined DMA solution, that any application can use to move data from one memory to another without having to resort to 20 abstraction layers to do it. SDXI is just about moving data between memory banks.

Today, this is all within one system/server, but as CXL matures and more and more hardware starts supporting CXL 2 and 3, shared memory between servers will become more pervasive all on a CXL memory interface.

Keith tried bringing it home to moving data between containers or VMs and all that’s possible today within the same memory and sometime in the future between shared memory and local memory. 

Memory to memory transfers have to be done securely. It’s not like accessing memory from some other process hasn’t been frought with security exposures in the past. And Dr. J assured me that SDXI was built from the ground up with security considerations front and center.

To bring it all back home. SNIA has always been and always will be concerned with data. Whether that data resides on storage, memory or god forbid, in transit somewhere over a network. Keith went as far as to say that the network was storage, I felt that was a step too far.

Dr. J Metz, Technical Director of Systems at AMD, Chair of SNIA BoD

J is the Chair of SNIA’s (Storage Networking Industry Association) Board of Directors and Technical Director for Systems Design for AMD where he works to coordinate and lead strategy on various industry initiatives related to systems architecture. Recognized as a leading storage networking expert, J is an evangelist for all storage-related technology and has a unique ability to dissect and explain complex concepts and strategies. He is passionate about the innerworkings and application of emerging technologies.

J has previously held roles in both startups and Fortune 100 companies as a Field CTO,  R&D Engineer, Solutions Architect, and Systems Engineer. He has been a leader in several key industry standards groups, sitting on the Board of Directors for the SNIA, Fibre Channel Industry Association (FCIA), and Non-Volatile Memory Express (NVMe). A popular blogger and active on Twitter, his areas of expertise include NVMe, SANs, Fibre Channel, and computational storage.

J is an entertaining presenter and prolific writer. He has won multiple awards as a speaker and author, writing over 300 articles and giving presentations and webinars attended by over 10,000 people. He earned his PhD from the University of Georgia.

141: GreyBeards annual 2022 wrap-up podcast

Well it has been another year and time for our annual year end wrap up. Since Covid hit, every year has certainly been interesting. This year we have seen the start of back in person conferences which was a welcome change from the covid lockdown. We are very glad to start seeing everybody again.

From the tech standpoint, the big news this year was CXL. As everyone should recall, CXL is a new-ish PCIe hardware and protocol that supports larger memory sitting out on a PCIe bus and in the future shared memory between servers. All this is to enable a new wave of memory based computing. We spent probably half our time discussing CXL and it’s impact on IT.

The other major topic was the Cloud Native ecosystem. In the past all we talked about was K8s but nowadays the ecosystem that surrounds it is almost as important as K8s itself. The final topic was a bit of a shock earlier this year and yes it was the Broadcom’s acquisition of VMware. Jason and I spend our Explore podcast talking about it (see our 137: VMware Explore wrap-up). Keith has high hopes that the EU will shut it down but the jury’s still out on that one. Listen to the podcast to learn more.

As for CXL, it turns out that AMD have just released full support for CXL hardware and protocols with their latest round of CPU chips. But the new AMD CPUs only support DDR5 memory, (something about there’s only so much logic one can fit on a chip…) which means all those DDR4 DIMs out in the wild need somewhere to land. CXL could supply a new lease on life for DDR4 DIMs.

And it’s not just about shared memory or increased memory sizes, CXL can also provide a tiered memory hierarchy, with gobs of flash behind memory DIMs (see: 136: FMS2022 wrap up …) So, now its no longer a TB or ten of server memory but potentially 100s of TBs. What this means for SAP HANNA, AWS Aurora and other heavy-memory solutions has yet to play out.

Cloud Native won. We see this in the increasing adoption of containers and K8s in the enterprise, cloud and just about anywhere IT happens these days. But the ecosystem surrounding K8s is chaos.

Over time, many of these ecosystem solutions will die off, be purchased, or consolidated but in the mean time, it’s entirely too confusing. Red Hat’s OpenShift is one answer and VMware’s Tanzu is another. And of course all the clouds have their own K8s packaged solution. But just to cover their bets, everyone also supports native K8s and just about every software package that works with it. So, K8s’s ecosystem is in a state of flux and may take time to become a stable set of tools useable by the enterprise IT.

Finally, Broadcom’s acquisition of VMware has everyone up in arms. Customers are concerned the R&D juggernaut that VMware has been, since its very beginning, will be jettisoned in favor of profits. And HCI vendors that always felt Dell EMC had an unfair advantage will all look at Broadcom in a similar light.

Keith says there’s a major difference in how USA regulators view an acquisition and how EU regulators view one. According to Keith, EU views acquisitions in how they help or hurt the customer. USA regulators view acquisitions on show they help or hurt the competition. Will have to wait and see how this all plays for Broadcom-VMware.

On the other hand, speaking of competition, Nutanix seems to be feeling the heat as well. Rumors are it’s up for sale. Who will want it and how the regulators view both of these acquisitions may be as interesting story for 2023

2023 looks to be another year of transition for enterprise IT. The cloud players all seem to be coming around to the view that they can’t be all things to all (IT) people. And the enterprise vendors are finally seeing some modicum of staying power in the face of a relentless push to the cloud. How this plays out over the next few years will be of major interest to everybody.

Happy New Year from the GreyBeards!

Keith Townsend, The CTO Advisor

Keith Townsend (@CTOAdvisor) is a IT thought leader who has written articles for many industry publications, interviewed many industry heavyweights, worked with Silicon Valley startups, and engineered cloud infrastructure for large government organizations. Keith is the co-founder of The CTO Advisor, blogs at Virtualized Geek, and can be found on LinkedIN.

Jason Collier, Principal Member of Technical Staff, AMD

Jason Collier (@bocanuts) is a long time friend, technical guru and innovator who has over 25 years of experience as a serial entrepreneur in technology. He was founder and CTO of Scale Computing and has been an innovator in the field of hyperconvergence and an expert in virtualization, data storage, networking, cloud computing, data centers, and edge computing for years. He’s on LinkedIN.

136: Flash Memory Summit 2022 wrap-up with Tom Coughlin, President, Coughlin Assoc.

We have known Tom Coughlin (@thomascoughlin), President, Coughlin Associates for a very long time now. He’s been an industry heavyweight almost as long as Ray (maybe even longer). Tom has always been very active in storage media, storage drives, storage systems and memory as well as active in the semiconductor space. All this made him a natural to perform as Program Chair at Flash Memory Summit (FMS)2022, so it’s great to have on the show to talk about the conference.

Just prior to the show, Micron announced that they had achieved 232 layer 3D NAND(in sampling methinks). Which would be a major step on the roadmap to higher density NAND. Micron was not at the show, but held an event at Levi stadium, not far from the conference center.

During a keynote, SK Hynix announced they had achieved 238 layer NAND, just exceeding Micron’s layer count. Other vendors at the show promised more layers as well but also discussed different ways other than layer counts to scale capacity, such as shrinking holes, moving logic, logical (more bits/cell) scaling, etc. PLC (5 bits/cell) was discussed and at least one vendor mentioned 6LC (not sure there’s a name yet but HxLC maybe?). Just about any 3D NAND is capable of logical scaling in bits/cell. So 200+ layers will mean more capacity SSDs over time.

The FMS conference seems to be expanding beyond Flash into more storage technologies as well as memory systems. In fact they had a session on DNA storage at the show.

In addition, there was a lot of talk about CXL, the new shared memory standard which supports shared memory over PCIe at FMS2022. PCIe is becoming a near universal connection protocol and is being used for 2d scaling of chips as a chip to chip interconnect as well as distributed storage and shared memory interconnect.

The CXL vision is that servers will still have DDR DRAM memory but they can share external memory systems. With shared memory systems in place memory, memory could be pooled and aggregated into one large repository which could then be carved up and parceled out to servers to support the workload dejour. And once those workloads are done, recarved up for the next workload to come. Almost like network attached storage only in this world its network attached memory.

Tom mentioned that CXL is starting to adopting other memory standers such as the Open Memory Interface (OMI) which has also been going on for a while now.

Moreover, CXL can support a memory hierarchy, which includes different speed memories such as DRAM, SCM, and SSDs. If the memory system has enough smarts to keep highly active data in the highest speed devices, an auto-tiering, shared memory pool could provide substantial capacities (10s-100sTB) of memory at a much reduced cost. This sounds a lot like what was promised by Optane.

Another topic at the show was Software Enabled/Defined Flash. There are a few enterprise storage vendors (e.g., IBM, Pure Storage and Hitachi) that design their own proprietary flash devices, but with SSD vendors coming out with software enabled flash, this should allow anyone to do something similar. Much more to come on this. Presumably, the hyper-scalers are driving this but having software enabled flash should benefit the entire IT industry.

The elephant in the room at FMS was Intel’s winding down of Optane. There were a couple of the NAND/SSD vendors talking about their “almost” storage class memory using SLC and other NAND tricks to provide Optane like performance/endurance using NAND storage.

Keith mentioned a youtube clip he saw where somebody talked about an Radeon Pro SSG ( (AMD GPU that had M.2 SSDs attached to it). And tried to show how it improved performance for some workloads (mostly 8k video using native SSG APIs). He replaced the old M.2 SSDs with newer ones with more capacity which increased the memory but it still had many inefficiencies and was much slower than HBM2 memory or VRAM. Keith thought this had some potential seeing as how in memory databases seriously increase performance but as far as I could see the SSG and it’s moded brethren died before it reached that potential.

As part of the NAND scaling discussion, Tom said one vendor (I believe Samsung) mentioned that by 2030, with die stacking and other tricks, they will be selling an SSD with 1PB of storage behind it. Can’t wait to see that.

By the way, if you are an IEEE member and are based in the USA, Tom is running for IEEE USA president this year, so please vote for him. It would be nice having a storage person in charge at IEEE.

Thomas Coughlin, President Coughlin Associates

Tom Coughlin, President, Coughlin Associates is a digital storage analyst and business and technology consultant. He has over 40 years in the data storage industry with engineering and senior management positions at several companies. Coughlin Associates consults, publishes books and market and technology reports (including The Media and Entertainment Storage Report and an Emerging Memory Report), and puts on digital storage-oriented events.

He is a regular storage and memory contributor for forbes.com and M&E organization websites. He is an IEEE Fellow, Past-President of IEEE-USA, Past Director of IEEE Region 6 and Past Chair of the Santa Clara Valley IEEE Section, Chair of the Consultants Network of Silicon Valley and is also active with SNIA and SMPTE.

For more information on Tom Coughlin and his publications and activities go to