Analyzing Starfield’s Efficiency on Nvidia’s 4090 and AMD’s 7900 XTX – Chips and Cheese
Bethesda has a historical past of constructing demanding AAA video games set in immersive open worlds. Starfield is the most recent, and might greatest be described as Skyrim in area. Open world video games put heavy calls for on builders and gaming {hardware} as a result of the elevated scope dramatically will increase issue for the whole lot concerned. Whereas Starfield gameplay seems comparatively bug free, its graphics efficiency has attracted consideration as a result of AMD playing cards are inclined to punch above their weight. With that in thoughts, let’s look at a specific scene.
We analyzed this scene utilizing Nvidia’s Nsight Graphics and AMD’s Radeon GPU Profiler to get some perception into why Starfield performs the way in which it does. On the Nvidia aspect, we coated the final three generations of playing cards by testing the RTX 4090, RTX 3090, and Titan RTX. On AMD, we examined the RX 7900 XTX. The i9-13900K was used to gather knowledge for all of those GPUs.
Particular thanks goes to Titanic for swapping GPUs and accumulating knowledge.
Excessive Stage
Starfield options quite a lot of compute. Pixel shaders nonetheless take a big period of time, as a result of we ran the sport at 4K to make it GPU sure. Vertex shaders and different geometry associated stuff present up too, notably close to the beginning of the timeline.
If we line up the occupancy timelines from Radeon Graphics Profiler and Nvidia’s Nsight Graphics, we are able to see particular person occasions line up throughout the GPUs. AMD and Nvidia swap colours for pixel and vertex shaders, however in any other case it’s fairly straightforward to correlate occasions.
We’ll be analyzing the three longest period calls as a result of digging by way of every of the ~6800 occasions can be impractical.
Longest Period Compute Shader
Our first goal might be a Dispatch (compute) name that happens close to the center of the body, indicated with a purple field:
The decision takes simply over 1.1 milliseconds on Nvidia and AMD’s quickest gaming GPUs. Whereas AMD’s 7900 XTX lands behind Nvidia’s RTX 4090, the 2 GPUs are nearer than you’d count on given Nvidia’s a lot bigger shader array. Older playing cards take twice as lengthy, and Turing particularly suffers.
By way of parallelism, the dispatch name operates on 6.8 million gadgets, that are organized into 213 thousand wavefronts (analogous to a CPU thread) on RDNA 3. An RTX 4090 working at max occupancy would be capable to observe 6144 waves, whereas an RX 7900 XTX would observe 3072. There’s greater than sufficient work to go round for each GPUs.
Whereas it is a compute shader, it seems to be doing texture processing of some kind, as a result of it incorporates a good variety of texture sampling directions.
Proper earlier than the shader exits, it writes to textures as effectively:
With out realizing extra, I think this name is utilizing compute to generate a texture that’s used afterward within the rendering pipeline.
Shader {Hardware} Utilization
RGP and Nsight Graphics each report utilization with respect to {hardware} maximums, so it’s helpful to consider this on the SIMD or SMSP stage. AMD’s RDNA 3 GPUs are constructed from Workgroup Processors (WGPs). Every WGP has 4 SIMDs, every of which has its personal register recordsdata, execution items, and a scheduler that may observe as much as 16 threads. By threads, I imply threads within the CPU sense the place every thread has an impartial instruction pointer. You may consider a SIMD as having 16-way SMT.
Nvidia GPUs are constructed from Streaming Multiprocessors (SMs), every of that are divided into 4 SM Sub-Partitions (SMSPs). Like a RDNA SIMD, every SMSP has a register file, execution items, and a scheduler. Ada and Ampere’s SMSPs can observe as much as 12 threads, whereas Turing’s can observe as much as eight.
On AMD’s RDNA 3, the vector ALU (VALU) metric is crucial, as vector computations are what GPUs are made for. The scalar ALU (SALU) pipe is used to dump computations that apply throughout the wave, like handle technology or management circulate operations. RDNA 3 enjoys superb vector utilization. Scalar ALUs do a very good job of offloading computation, as a result of the vector items would see fairly excessive load in the event that they needed to deal with scalar operations too.
On Nvidia GPUs, the SM challenge metric is an effective place to start out, because it signifies how typically the scheduler was capable of finding an unstalled thread to challenge an instruction from. Every SMSP can challenge one instruction per cycle, so the utilization additionally corresponds to IPC on the SMSP stage. Ampere and Ada see good utilization, although inferior to AMD’s. Turing’s scenario is medicore. 26.6% challenge utilization is considerably decrease than what we see on the opposite examined GPUs.
Nvidia additionally organizes execution items into completely different pipes, suggesting an Ada or Ampere SMSP appears one thing like this:
I wouldn’t put an excessive amount of weight into utilization for particular person pipes except it’s excessive sufficient to trigger issues with challenge utilization. On this case, utilization is low sufficient for particular person pipes that they’re doubtless not a bottleneck.
Occupancy
To elucidate the utilization figures, we are able to begin by occupancy. GPUs use thread stage parallelism to maintain themselves fed as a result of they don’t have out-of-order execution. If one thread stalls, they swap to a different one. Extra threads imply a greater likelihood of preserving the execution items fed, similar to how SMT on CPUs helps conceal latency. Earlier, I discussed that AMD SIMDs had been mainly sub-cores with 16-way SMT, whereas Nvidia’s SMSPs had 12 or 8-way SMT for Ampere/Ada or Turing. Occupancy refers to what number of of these SMT threads are in use.
Excessive occupancy isn’t required to get good {hardware} utilization. Simply as CPUs can obtain max throughput with only one SMT thread energetic, the entire GPUs listed here are able to maxing out their vector items with one thread in every SMSP or SIMD. Nevertheless, cache entry latency could be very excessive on GPUs, so larger occupancy typically correlates with higher utilization.
Not one of the graphics architectures examined right here obtain max occupancy, however AMD’s RDNA 3 enjoys a big benefit. Register file capability is accountable right here. Not like CPUs, these GPUs dynamically allocate register file capability between threads, and what number of threads you possibly can have energetic typically is dependent upon what number of registers every thread is utilizing. AMD’s compiler selected to allocate 132 vector registers for this code, which rounds as much as 144 registers as a result of RDNA 3 allocates registers in blocks of 24. Every thread due to this fact requires 18.4 KB of vector registers (32 lanes * 4 bytes per register * 144 registers). A SIMD’s register file gives 192 KB of capability – sufficient to carry state for 10 threads.
Every Nvidia SIMD has 64 KB of registers, and Nsight signifies waves couldn’t be launched to the shader array as a result of no registers had been out there. Nvidia’s compiler doubtless gave every wave 128 registers, or 16 KB of register file capability. Register allocation doesn’t differ a lot between AMD and Nvidia, however Nvidia’s a lot smaller register file means its architectures can’t preserve as a lot work in flight per SIMD lane.
Caching
Conserving extra work in flight helps a GPU deal with latency, however slicing down latency helps too. Identical to CPUs, GPUs have advanced to characteristic complicated multi-level cache hierarchies. VRAM applied sciences haven’t saved up with advances in GPU efficiency. Caches thus assist GPUs preserve their execution items fed by decreasing how typically they wait on VRAM.
RDNA 3, Ampere, and Ada all get pleasure from excessive first stage cache hitrates. L2 hitrates are excessive as effectively. Based on Nsight, the shader used 9.6%, 6.6%, and 11.4% of L2 bandwidth on the RTX 4090, RTX 3090, and Titan RTX respectively. We’re due to this fact principally L1 latency, and a little bit of L2 latency. In prior articles, we measured L1 latency utilizing vector accesses. Nevertheless, latency there may be in all probability not consultant of latency seen on this shader as a result of RDNA 3 shader disassembly reveals there are quite a lot of texture accesses. Texture filtering incurs further latency, which might be troublesome to cover even when knowledge comes from the primary stage of cache.
Nvidia’s Nsight Graphics lacks the power to indicate shader disassembly, however their scenario might be related.
The takeaway from this shader is that AMD’s RDNA 3 structure is best set as much as feed its execution items. Every SIMD has 3 times as a lot vector register file capability as Nvidia’s Ampere, Ada, or Turing SMSPs, permitting larger occupancy. That in flip offers RDNA 3 a greater likelihood of hiding latency. Whereas L1 hitrates are good, excessive occupancy nonetheless issues as a result of texture sampling incurs larger latency than plain vector accesses. However though AMD is best at feeding its execution items, Nvidia’s RTX 4090 merely has a a lot bigger shader array, and nonetheless manages to drag forward.
Longest Period Pixel Shader
Trendy video games proceed to make use of rasterization as a basis (even when they assist raytracing), and pixel shaders are a vital a part of the rasterization pipeline. They’re resposible for calculating pixel coloration values and passing them right down to the render output items (ROPs). Increased resolutions imply extra pixels, which imply extra pixel shader work.
The pixel shader in query was dispatched with 8.2 million invocations, which covers all the 4K display. Once more, there’s greater than sufficient parallelism to fill the shader array on all of those GPUs. Nvidia’s RTX 4090 comes out prime, ending the shader in 0.93 ms. AMD’s RX 7900 XTX is a tad slower at 1.18 ms. Prior technology GPUs are considerably behind with the 3090 taking 2.08 ms, and the Titan RTX taking 4.08 ms.
AMD’s RX 7900 XTX sees exceptionally excessive vector ALU utilization. This isn’t essentially a very good factor as a result of such excessive utilization signifies a compute bottleneck. Nvidia’s Ampere and Ada Lovelace architectures see a extra affordable 60% utilization. They’re not under-fed, however aren’t compute sure both.
Turing once more has decrease utilization, however in isolation 35.9% will not be dangerous. CPU execution ports typically see related utilization in excessive IPC workloads. Turing isn’t feeding itself badly, but it surely’s simply not doing in addition to extra trendy Nvidia GPUs.
Once more, Nsight reveals occupancy restricted by register file capability. Ada and Ampere do an excellent job of preserving their execution items fed, contemplating every SMSP on common solely had 4-5 threads to select from to cover latency.
AMD’s compiler used 92 registers, which rounds as much as 96 once we take into account RDNA 3’s 24 register allocation granularity. That’s 12 KB of registers, which completely makes use of 192 KB SIMD vector register file. There’s a very good likelihood Nvidia’s compiler selected the identical variety of registers, as a result of theoretical occupancy seems to be 5 threads per SMSP. 5 threads * 12 KB of registers per thread comes out to 60 KB of register file capability used. Nsight shows achieved occupancy slightly than theoretical occupancy. Every SMSP may in all probability observe 5 threads, however typically tracks barely much less as a result of the work distribution {hardware} may not launch a brand new wave instantly when one finishes.
You may see the identical phenomenon on RDNA 3, however RDNA 3 does appear barely higher at sending out work sooner. AMD’s decoupled frontend clock may assist with this, because it lets the work distribution {hardware} run at larger clocks than the shader array.
First stage caches carry out fairly effectively. Nvidia enjoys barely larger L1 hitrates as a result of every SM has a bigger L1 than AMD’s 32 KB L0. Nvidia does allocate L1 cache and shared reminiscence (scratchpad) capability out of the identical block of SRAM, however that’s not a problem right here. AMD didn’t use any scratchpad reminiscence, and I think Nvidia didn’t both, so this shader in all probability acquired the utmost L1 allocation.
AMD’s decrease L0 cache hitrate is mitigated by the 256 KB L1 mid-level cache. L1 hitrate by itself will not be spectacular, but it surely does imply AMD’s L2 doubtless sees much less site visitors than Nvidia’s. Cumulative L0 and L1 hitrate can be about 94.3% if we approximate by assuming L1 general hitrate is an effective illustration of L1 knowledge hitrate. It ought to be shut sufficient as a result of the L1 was principally serving L0 miss site visitors.
The takeaway from this shader is that AMD is ready to obtain very excessive utilization because of very excessive occupancy. In actual fact, utilization is so excessive that AMD is compute sure. Nvidia {hardware} does effectively on this shader, however not fairly as effectively as a result of they once more don’t have sufficient register file capability to maintain as a lot work in flight. Nevertheless, Nvidia’s RTX 4090 nonetheless wins in absolute phrases as a result of it has so many extra SMs.
Second Longest Compute Shader
Lastly, let’s have a look at one other compute shader. This one comes up third if we kind all occasions by period, so it’s the third most vital shader general.
This compute shader launches 2 million invocations, cut up into 32,400 wavefronts on RDNA 3. It’s a really fascinating case as a result of AMD’s 7900 XTX truly beats Nvidia’s RTX 4090. Within the earlier two shaders, the 7900 XTX merely acquired nearer to Nvidia’s flagship than anticipated.
Much more apparently, AMD doesn’t benefit from the occupancy benefit we noticed earlier than. Nevertheless, the scenario is kind of sophisticated as a result of AMD opted to run this shader in wave64 mode, in distinction to the wave32 mode used earlier than. Their compiler allotted a reasonably reasonable 93 (rounding as much as 96) vector registers per thread. Wave64 means every thread is mainly executing 2048-bit vector directions, double pumped by way of every SIMD’s 1024-bit extensive execution items. Sure frequent directions can execute with full 1-per-cycle throughput even in wave64 mode, because of RDNA 3’s twin 32-wide execution items. Nevertheless, utilizing 64-wide vectors require twice as a lot register file capability as 32-wide ones, so AMD’s SIMDs solely have sufficient register file capability to trace eight threads at a time.
Nvidia can’t use a wave64 mode, and the inexperienced group’s compiler doubtless allotted fewer registers per thread as effectively. If we glance simply at occupancy, there’s not quite a lot of distinction between the GPUs. Ada has a slight benefit and possibly has a theoretical register-limited occupancy of 9 warps per SMSP. Nevertheless, now we have to do not forget that every wave64 thread on AMD has twice the SIMD width.
RDNA 3 enjoys very excessive utilization, and steps proper on the road with reference to being thought of compute sure. For reference, Nvidia considers a workload sure by a specific unit’s throughput if it’s achieving over 80% of theoretical performance. AMD doesn’t want larger occupancy. Nvidia’s GPUs see far decrease utilization. The RTX 4090 sits just under 30%. Ampere and Turing are in a reasonably good place, although notably Turing sees very excessive FMA pipe utilization. Turing’s SMSPs can solely challenge FP32 operations each different cycle as a result of every SMSP solely has a single 16-wide FP32 pipe. Turing may grow to be compute restricted if it hit different bottlenecks first. That different bottleneck seems to be the L2 cache. L2 site visitors was low sufficient to be a non-factor within the different two shaders and throughout a lot of the body. However this compute name is completely different.
All GPUs cross the 80% threshold for being thought of L2 bandwidth sure. Nevertheless, the AMD determine comes with a caveat as a result of RGP shows request counts as an alternative of a p.c of theoretical throughput. I approximated throughput utilization by multiply hit rely by 128B cacheline measurement (giving 4.844 TB/s of L2 site visitors), and divided by our previously measured figure of 5.782 TB/s for RDNA 3’s L2 cache. Measured figures normally can’t get near 100% utilization, and a better have a look at the BW scaling graph signifies it was in all probability measuring latency-limited bandwidth. Subsequently, the 83.78% determine ought to be handled as an overestimate. I wouldn’t be shocked if actual L2 bandwidth utilization was effectively below 80% on RDNA 3.
Throughout Nvidia’s GPUs, RTX 4090 appears particularly bandwidth sure. Zooming in signifies the shader spends a lot of the time pushing effectively past 90% L2 bandwidth utilization.
Subsequently, excessive occupancy doesn’t assist Nvidia get higher utilization than AMD. Earlier, I discussed that larger occupancy doesn’t essentially result in higher utilization, and that is one such case. Loading all SMT threads on a CPU gained’t provide help to in bandwidth-limited eventualities, and GPUs aren’t any completely different.
This compute shader has a bigger scorching working set than the prior ones, and L1 hitrate is decrease throughout all three GPUs. Nvidia’s GPUs have a reasonably laborious time preserving accesses inside their L1 caches. AMD someway enjoys the next hitrate for its small 32 KB L0 cache, although the bigger 256 KB L1 barely enters the image with a measly 13.5% hitrate.
L2 caches are massive sufficient to catch the overwhelming majority of L1 misses throughout all examined GPUs. Earlier, I assumed Nvidia’s RTX 4090 had enough L2 bandwidth to handle most workloads, so Nvidia’s easier two-level cache hierarchy was justified. This shader is an exception, and L2 bandwidth limits stop Nvidia’s a lot bigger RTX 4090 from beating the RX 7900 XTX.
Feedback on GPU Utilization
Like most video games, Starfield is a posh workload that sees completely different calls for all through a body. Within the two longest period shaders we checked out, AMD was in a position to leverage its bigger vector register file to maintain extra work in flight per SIMD. That in flip gave it a greater likelihood of hiding cache and execution latency.
Nevertheless, amount has a top quality all of its personal, and it’s laborious to argue with 128 SMs sitting on a big 608 mm2 die. AMD could also be higher at feeding its execution items, however Nvidia doesn’t do a nasty job. 128 reasonably effectively fed SMs nonetheless find yourself forward of 48 very effectively fed WGPs, letting Nvidia preserve the 4K efficiency crown. AMD’s 7900 XTX makes use of simply 522 mm2 of die space throughout all its chiplets. To nobody’s shock, it will possibly’t match the throughput of Nvidia’s monster even when we take into account wave64 or wave32 twin challenge.
In AMD’s favor, they’ve a really excessive bandwidth L2 cache. As the primary multi-megabyte cache stage, the L2 cache performs a really important function and sometimes catches the overwhelming majority of L0/L1 miss site visitors. Nvidia’s GPUs grow to be L2 bandwidth sure within the third longest shader, which explains a little bit of why AMD’s 7900 XTX will get as shut because it does to Nvidia’s a lot bigger flagship. AMD’s win there’s a small one, however seeing the a lot smaller 7900 XTX pull forward of the RTX 4090 in any case will not be in keeping with anybody’s expectations. AMD’s cache design pays off there.
Remaining Phrases
In abstract there’s no single clarification for RDNA 3’s relative overperformance in Starfield. Increased occupancy and better L2 bandwidth each play a task, as does RDNA 3’s larger frontend clock. Nevertheless, there’s actually nothing mistaken with Nvidia’s efficiency on this recreation, as some feedback across the web may recommend. Decrease utilization is by design in Nvidia’s structure. Nvidia SMs have smaller register recordsdata and might preserve much less work in flight. They’re naturally going to have a tougher time preserving their execution items fed. Chopping register file capability and scheduler sizes helps Nvidia scale back SM measurement and implement extra of them. Nvidia’s design comes out prime with kernels that don’t want quite a lot of vector registers and revel in excessive L1 cache hitrates.
If we have a look at the body as an entire, the RTX 7900 XTX rendered the body in 20.2 ms, for just below 50 FPS. Nvidia’s RTX 4090 took 18.1 ms, for 55.2 FPS. A win is a win, and validates Nvidia’s technique of utilizing an enormous shader array even when it’s laborious to feed. Going ahead, AMD will want extra compute throughput in the event that they wish to contend for the highest spot.
In the event you like our articles and journalism, and also you wish to assist us in our endeavors, then take into account heading over to our Patreon or our PayPal if you wish to toss a couple of dollars our means. If you want to speak with the Chips and Cheese employees and the folks behind the scenes, then take into account becoming a member of our Discord.