fix(compiler): carry vector operands through inlining and CSE

Two passes walk an instruction's operands by hand and end the match
with a catch-all, and the vector family fell into it. Both were live
miscompiles, found writing a tensor kernel in ZynML.

Inlining substitutes the id an instruction defines so a cloned body
lands on fresh values, and its list of instructions that define
something named none of the vector ops. An inlined `vector_splat` kept
the id it had in the callee while every use of it moved to the fresh
one, so `vstore` wrote a value nothing had computed. A store inside a
called function silently did nothing and the kernel read back zero.

CSE rewrites the uses of an instruction it eliminates, over thirteen
kinds and no vector ones. Eliminating a redundant `extractvalue` left a
`vload` pointing at an id that no longer existed, which Cranelift met
as "arg not in value_map".

`HirInstruction::replace_uses` covers all nine ops and is the one to
reach for. These went unnoticed because vectorised code stays inside
the function that produced it, so neither pass had met a vector op
until one was written by hand.

Author: darmie

crates/compiler/src/cse.rs

@@ -568,6 +568,32 @@ fn rewrite_inst_operands(inst: &mut HirInstruction, map: &impl Fn(&mut HirId) ->
         HirInstruction::Unary { operand, .. } => m!(operand),
         HirInstruction::Cast { operand, .. } => m!(operand),
         HirInstruction::Load { ptr, .. } => m!(ptr),
+        // The vector family. Leaving these out let this pass delete an
+        // instruction and rewrite every use of it except the ones a
+        // vector op held, so a `vload` kept pointing at an id nothing
+        // defined any more.
+        HirInstruction::VectorSplat { scalar, .. } => m!(scalar),
+        HirInstruction::VectorExtractLane { vector, .. } => m!(vector),
+        HirInstruction::VectorInsertLane { vector, scalar, .. } => {
+            m!(vector);
+            m!(scalar);
+        }
+        HirInstruction::VectorHorizontalReduce { vector, .. } => m!(vector),
+        HirInstruction::VectorLoad { ptr, .. } => m!(ptr),
+        HirInstruction::VectorStore { value, ptr, .. } => {
+            m!(value);
+            m!(ptr);
+        }
+        HirInstruction::VectorUnaryOp { operand, .. } => m!(operand),
+        HirInstruction::VectorMinMax { left, right, .. } => {
+            m!(left);
+            m!(right);
+        }
+        HirInstruction::VectorDot { acc, a, b, .. } => {
+            m!(acc);
+            m!(a);
+            m!(b);
+        }
         HirInstruction::Store { value, ptr, .. } => {
             m!(value);
             m!(ptr);

crates/compiler/src/inline.rs

@@ -1341,7 +1341,19 @@ fn substitute_operands(inst: &mut HirInstruction, subs: &HashMap<HirId, HirId>)
         | HirInstruction::InsertValue { result, .. }
         | HirInstruction::Alloca { result, .. }
         | HirInstruction::Select { result, .. }
-        | HirInstruction::Atomic { result, .. } => map_result(result),
+        | HirInstruction::Atomic { result, .. }
+        // Every vector instruction that defines a value. Leaving these
+        // out let the definition keep the id it had in the callee while
+        // its uses moved to the fresh one, so an inlined
+        // `vstore v, p` wrote a value nothing had computed.
+        | HirInstruction::VectorSplat { result, .. }
+        | HirInstruction::VectorExtractLane { result, .. }
+        | HirInstruction::VectorInsertLane { result, .. }
+        | HirInstruction::VectorHorizontalReduce { result, .. }
+        | HirInstruction::VectorLoad { result, .. }
+        | HirInstruction::VectorUnaryOp { result, .. }
+        | HirInstruction::VectorMinMax { result, .. }
+        | HirInstruction::VectorDot { result, .. } => map_result(result),
         HirInstruction::Call { result, .. } | HirInstruction::IndirectCall { result, .. } => {
             if let Some(r) = result {
                 map_result(r);