Constant Folding Feature Analysis【免费下载链接】geGEGraph Engine是面向昇腾的图编译器和执行器提供了计算图优化、多流并行、内存复用和模型下沉等技术手段加速模型执行效率减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge1 Feature OverviewConstant folding is a core compilation optimization in GE graph compiler. Its core idea: during graph compilation stage, identify operator nodes with all inputs as constants, complete computation on host side ahead of time, write computation results back as new constant nodes, thereby eliminating unnecessary runtime computation overhead.GE constant folding is not simply constant expression evaluation, but forms a complete optimization system containing dimension computation, empty tensor replacement, potential constant marking, delayed effecting mechanisms. It runs through multiple compilation stages including graph preprocessing (Prepare), optimization stage 1 (OptimizeStage1), optimization stage 2 (OptimizeStage2), before auto-fusion (BeforeAutofuse).1.1 Problems SolvedIn deep learning computation graphs, many node inputs are determinable at compilation time (like model weights, hyperparameters, Shape-related operations). If these nodes remain to runtime execution on device, two problems arise:Unnecessary computation overhead: Shape, Rank, Size pure meta-information operations can completely complete at compilation timeBlock subsequent optimization: Unfolded constant nodes block dead code elimination, common subexpression elimination optimizations1.2 Functional ScopeConstant folding feature contains following capabilities:CapabilityDescriptionStandard constant foldingReplace operators with all constant inputs as constant nodesDimension computation foldingCompile-time evaluation for Shape/Reshape/Transpose dimension operationsDimension adjustment foldingIn-place optimize and remove dimension-changing operators like ExpandDimsEmpty tensor replacementReplace operators with empty tensor output as empty constant nodesPotential constant markingMark nodes that currently have partial non-constant inputs but may become all-constant in futurePotential constant effectingAfter graph traversal ends, uniformly execute folding on already marked potential constant nodes2 User Usage Scenarios2.1 Offline Compilation Scenario (atc)When users use atc tool to compile ONNX/PB models as OM files, constant folding defaults to enable at O1 and above optimization levels. Users can control switch through configuration parameters:atc --modelmodel.onnx --outputmodel --framework5 \ --ge.oo.levelO1 --ge.oo.constantFoldingtrueTypical benefit scenario: Model contains large amount of auxiliary computations for dynamic Shape inference (like Shape→Gather→Concat→Reshape chain). These can completely pre-compute in static Shape scenario. Constant folding can eliminate them all.2.2 Online Compilation Scenario (aclgrphBuildModel)When using ACL Graph Builder API to build models, control throughaclgrphBuildInitializeoraclgrphBuildModelconfiguration parameters:// Global level configuration std::mapge::AscendString, ge::AscendString global_options { {ge::ir_option::OO_LEVEL, O1}, {ge::ir_option::OO_CONSTANT_FOLDING, true} }; aclgrphBuildInitialize(global_options); // Graph level configuration (higher priority) std::mapge::AscendString, ge::AscendString build_options { {ge::ir_option::OO_CONSTANT_FOLDING, true} }; aclgrphBuildModel(graph, build_options, modelBufferData);2.3 Debugging ScenarioWhen users suspect constant folding causes result abnormality, can disable this optimization for comparison verification:{ge::ir_option::OO_CONSTANT_FOLDING, false}After disabling, Size, Shape, ShapeN, Rank operators will not be folded and deleted, still execute on device at runtime.GeDeletedOpmechanism inge_deleted_op.ccwill give clear error indications for these operators, helping users locate problems.2.4 User Specified Skip FoldingFramework side (like TensorFlow_grappler_do_not_removeattribute) or users can prevent specific nodes from being constant folded by setting node attribute_do_not_constant_folding. This provides fine control means for scenarios requiring preserving specific nodes.3 External Interfaces3.1 Configuration ParametersParameter KeyParameter ValueConfiguration EntryDescriptionge.oo.constantFoldingtrue/falseaclgrphBuildInitialize, aclgrphBuildModel, atcControl constant folding optimization switchge.oo.levelO1/O2/O3Same as aboveOptimization level, O1 and above default enable constant foldingParameter definition located atinc/graph_metadef/external/ge_common/ge_api_types.h, constant nameOO_CONSTANT_FOLDING, actual configuration key isge.oo.constantFolding.Option registration located atbase/common/option_register.cc:REG_OPTION(OO_CONSTANT_FOLDING) .LEVELS(OoLevel::kO1) .DEFAULT_VALUES({{OoLevel::kO1, true}, {OoLevel::kO3, true}}) .CHECKER(OoInfoUtils::IsSwitchOptValueValid) .VISIBILITY({OoEntryPoint::kSession, OoEntryPoint::kIrBuild, OoEntryPoint::kAtc}) .SHOW_NAME(OoEntryPoint::kAtc, oo_constant_folding, OoCategory::kModelTuning)Option defaults to enable at O1 and O3 optimization levels, supports Session, IR Build and ATC three entry configurations.3.2 Node Attribute InterfacesAttribute NameFunctionSetterATTR_NO_NEED_CONSTANT_FOLDINGMark node no need for constant foldingGE internal PassATTR_NAME_DO_NOT_CONSTANT_FOLDINGMark user specified node not do constant foldingFramework Parser / UserATTR_NAME_POTENTIAL_CONSTMark node as potential constantGE constant folding PassATTR_NAME_POTENTIAL_WEIGHTStore potential constant weight valueGE constant folding PassATTR_NAME_POTENTIAL_WEIGHT_INDICESStore potential constant output indicesGE constant folding Pass_is_from_constant_foldingMark constant node produced by constant foldingGE constant folding PassATTR_NAME_IS_INSERTED_BY_GEMark node inserted by GE internallyGE internal PassAttribute definition located atinc/graph_metadef/graph/debug/ge_attr_define.h.3.3 Tool Class InterfaceGraphOptimizeUtility::ConstantFolding(located atcompiler/graph/manager/util/graph_optimize_utility.cc) provides single node level constant folding interface, for other modules (like weight compression judgmentWeightCompressJudge) to call as needed. This interface sequentially executes ConstantFoldingPass → DimensionComputePass → ReplaceWithEmptyConstPass.4 Specific Implementation4.1 Overall ArchitectureConstant folding adopts Pass chain execution mode, driven through GE Pass management framework. Core implementation concentrated incompiler/graph/passes/standard_optimize/constant_folding/directory, composed of 7 Passes and supporting infrastructure:┌──────────────────────────┐ │ BaseNodePass │ │ (compiler/graph/passes │ │ /base_pass.h) │ └──────────┬───────────────┘ │ ┌──────────▼───────────────┐ │ FoldingPass │ │ (folding_pass.h/cc) │ │ Folding basic operations:│ │ Create Const node, delete│ │ original node │ │ Reconnect data edges, │ │ preserve control edges │ └──────────┬───────────────┘ │ ┌───────────────┼───────────────┐ │ │ │ ┌──────────▼──────┐ ┌─────▼──────────┐ ┌──▼──────────────────┐ │ PotentialFolding │ │DimensionAdjust │ │ ReplaceWithEmpty │ │ Pass │ │ Pass │ │ ConstPass │ │ (potential_ │ │(dimension_ │ │(replace_with_ │ │ folding_pass) │ │ adjust_pass) │ │ empty_const_pass) │ └────────┬────────┘ └────────────────┘ └─────────────────────┘ │ ┌──────────┼──────────┐ │ │ ┌──────▼──────┐ ┌───────────▼──────────┐ │ Constant │ │ DimensionCompute │ │ FoldingPass │ │ Pass │ │ (constant_ │ │ (dimension_compute_ │ │ folding_ │ │ pass) │ │ pass) │ └──────────────────────┘ └─────────────┘4.2 Pass Inheritance SystemConstant folding Pass system adopts three-layer inheritance structure, progressively adding capabilities:BaseNodePass→ Defines basic framework for node-level Pass, including node traversal, re-pass (RePass), node deletion tracking mechanisms.FoldingPass→ Inherits BaseNodePass, implements core folding action (FoldingPass::Folding), including:Collect folded node downstream anchor connection relationships (GetIndexAndPeerInDataAnchors)Handle Switch/RefSwitch type input nodes (disconnect data edges, insert Identity node maintain control dependency)Create new Const node replace original node output (AddConstNodeToGraph)Connect new Const node to all downstream consumers of original nodeIsolate and delete original node and its input constant nodes becoming orphanedTransfer stream label (Stream Label) attribute to maintain stream planning correctnessPotentialFoldingPass→ Inherits FoldingPass, introduces potential constant concept and computation scheduling mechanism, unified orchestration byPotentialFoldingPass::Run.ConstantFoldingPass / DimensionComputePass / ReplaceWithEmptyConstPass→ Inherits PotentialFoldingPass, each implements different computation strategies and judgment logic.DimensionAdjustPass→ Directly inherits BaseNodePass, handles dimension adjustment operators (like ExpandDims), after completing computation on host side directly isolate and delete, not go through complete constant folding flow.PotentialConstTakenEffectPass→ Directly inherits FoldingPass, responsible for uniformly processing all already marked potential constant nodes after graph traversal ends.4.3 Core Pass Details4.3.1 ConstantFoldingPass — Standard Constant FoldingFile:compiler/graph/passes/standard_optimize/constant_folding/constant_folding_pass.h/ccThis is constant folding main entry Pass, processing nodes with all inputs as constants. Its execution flow:Pre-checkCheck if user set_do_not_constant_foldingattribute, if yes then skipCheck if node markedATTR_NO_NEED_CONSTANT_FOLDING, if yes then skipCheck if node is potential empty constant (all output Shape empty), if yes then hand to ReplaceWithEmptyConstPass handleInput VerificationConfirm all node inputs are constant nodes (throughOpDescUtils::GetConstInputNodeAndAnchor)If non-constant input exists but node marked as potential constant, then setneed_fold_ falseand return NOT_CHANGED (this round not fold, but preserve mark)Extract weight values from constant input nodes (throughOpDescUtils::GetWeightsFromNodes)Memory Priority Strategy CheckWhen MemoryPriority strategy configured, if input constant node Shape large (8) and shared by multiple downstream, then skip folding. Because folding copies constant data once, in memory constrained scenario not worthwhileComputation Execution (Two-level Fallback Strategy)First level: AICPU operator kernel(ComputeWithHostCpuKernel)Try throughaicpu_ascend_kernelengine get operator Host CPU implementationCreate operator instance throughOpKernelRegistry, execute byHostCpuEngineThis level supports widest operator types, runtime loadslibconstant_folding_ops.soSecond level: GE built-in kernel(ComputeWithBuiltInKernel)If AICPU does not support this operator, fallback to GE built-in Host KernelThroughKernelFactorylookup registered Kernel by operator type (folding_pass::GetKernelByType)GE built-in Kernel covers about 40 common operatorsFolding ReplacementAfter computation success, complete graph structure transformation byFoldingPass::FoldingNewly created Const node will be marked_is_from_constant_foldingtrue, for subsequent flow identificationPerformance StatisticsSeparately record AICPU kernel and GE built-in kernel folding time and call countSummary output performance trace log inGraphManager::OptimizeStage1_2Pass registration macro:REG_PASS_OPTION(ConstantFoldingPass).SWITCH_OPT(ge::OO_CONSTANT_FOLDING);This Pass controlled byge.oo.constantFoldingswitch.4.3.2 DimensionComputePass — Dimension Computation FoldingFile:compiler/graph/passes/standard_optimize/constant_folding/dimension_compute_pass.h/ccSpecifically handles dimension-related computation operations (like Shape, Reshape, Transpose). Difference from ConstantFoldingPass:Only uses GE built-in Kernel: Get Kernel throughfolding_pass::GetKernelByTypeand compute, not try AICPU kernelSupports mark but not fold mode: Can through constructor parameterneed_foldcontrol whether only do computation marking (in preprocessing stage run inneed_foldfalsemode, only mark potential constant not actually fold)Supports coordination with PotentialFoldingPass potential constant mechanismIn preprocessing stage (GraphPrepare::ComputeConstantShape), DimensionComputePass runs inneed_foldfalsemode, purpose is first use dimension computation determine Shape information, provide more accurate input for subsequent InferShape.Pass registration macro:REG_PASS_OPTION(DimensionComputePass).LEVELS(OoLevel::kO1).SWITCH_OPT(ge::OO_CONSTANT_FOLDING);4.3.3 DimensionAdjustPass — Dimension Adjustment FoldingFile:compiler/graph/passes/standard_optimize/constant_folding/dimension_adjust_pass.h/ccHandles ExpandDims operators adjusting dimension through constant parameters. Workflow:Get node original type, lookup corresponding GE built-in KernelCheck if unknown Shape, unknown Shape nodes skipCall KernelCompute(node)interface execute dimension adjustment computationDelete useless constant axis parameter input nodesThroughIsolateAndDeleteNodeisolate and delete node, and reconnect data edges by{0}IO mapping (only preserve data input direct connect data output)DimensionAdjustPass does not go through complete constant folding replacement flow (does not create Const node), but directly completes dimension inference then simplifies node as direct connect.Pass registration macro:REG_PASS_OPTION(DimensionAdjustPass).LEVELS(OoLevel::kO1).SWITCH_OPT(ge::OO_CONSTANT_FOLDING);4.3.4 ReplaceWithEmptyConstPass — Empty Tensor ReplacementFile:compiler/graph/passes/standard_optimize/constant_folding/replace_with_empty_const_pass.h/ccIdentifies operators with empty tensor output (Shape contains dimension 0, like[0, 3, 224, 224]), replaces as empty constant nodes.Exclusion rules (node types not replaced):Const/ConstantOp/FileConstant constant class nodesData class nodesNetOutput nodesControl flow operators (Switch, Merge, Enter, NextIteration, Exit, LoopCond etc.)Resource operators (Stack, StackPop, StackPush)Hcom collective communication operatorsNodes without output descriptionGE internally inserted nodes (ATTR_NAME_IS_INSERTED_BY_GEmark)This Pass also supportsneed_foldparameter control whether only do marking.Pass registration macro:REG_PASS_OPTION(ReplaceWithEmptyConstPass).LEVELS(OoLevel::kO1).SWITCH_OPT(ge::OO_CONSTANT_FOLDING);4.3.5 PotentialConstTakenEffectPass — Potential Constant EffectingFile:compiler/graph/passes/standard_optimize/constant_folding/potential_const_taken_effect_pass.h/ccThis is a special Pass, itsRunmethod is no-op (directly returns SUCCESS), actual work completes inOnFinishGraphcallback.Design intent: During graph traversal, some node partial inputs are not yet constants in current round (like from Shape inference intermediate results). DimensionComputePass etc. will mark as potential constant and cache weight. When graph traversal completes (all Pass executed), potential constant inputs may have been folded as constant in previous rounds. At this time PotentialConstTakenEffectPass uniformly scans all nodes marked as potential constant inOnFinishGraph, executes delayed folding.Processing flow:Traverse all nodes in graph, find nodes withATTR_NAME_POTENTIAL_CONSTmarkRead cached potential weight from attributes (ATTR_NAME_POTENTIAL_WEIGHT)If weight exists, callFoldingPass::Foldingexecute foldingIf weight missing, clear potential constant related attributes and record warningCollect all nodes needing re-pass, pass to next round Pass executionPass registration macro:REG_PASS_OPTION(PotentialConstTakenEffectPass).LEVELS(OoLevel::kO1).SWITCH_OPT(ge::OO_CONSTANT_FOLDING);4.4 Potential Constant MechanismPotential constant (Potential Const) is one of GE constant folding core innovations, solving current inputs not all constant but may become constant in future scenario.Related attributes and tool classes:ComponentLocationDescriptionConstantUtilsinc/graph_metadef/graph/utils/constant_utils.hPotential constant mark/query/clear tool classATTR_NAME_POTENTIAL_CONSTinc/graph_metadef/graph/debug/ge_attr_define.hMark node as potential constantATTR_NAME_POTENTIAL_WEIGHTSame as aboveCache potential constant weight valueATTR_NAME_POTENTIAL_WEIGHT_INDICESSame as aboveMark which output indices are potential constants_source_pass_of_potential_constpotential_folding_pass.ccRecord source Pass name, prevent cross Pass misoperationMechanism flow:DimensionComputePass / DimensionComputePass │ │ (node partial inputs non-constant) ▼ ComputePotentialWeight() → Computation success, get output weight → need_fold false (this round not fold) │ ▼ UpdatePotentialConstMark() → MarkPotentialConstWithPassName() → ConstantUtils::MarkPotentialConst() → Set _source_pass_of_potential_const │ ▼ (Multiple rounds InferShape ConstantFolding after, partial inputs folded as constant by other Pass) │ ▼ PotentialConstTakenEffectPass::OnFinishGraph() → Read ATTR_NAME_POTENTIAL_WEIGHT → FoldingPass::Folding() execute foldingIsCurPassSameWithSourcemethod inPotentialFoldingPassensures only Pass originally marking potential constant can update or clear that mark, avoiding interference between different Passes.4.5 Computation EngineConstant folding computation engine divides into two levels:4.5.1 GE Built-in Host KernelLocated atcompiler/host_kernels/, throughKernelFactoryregistration and creation.Kernelbase class defines three Compute interfaces:Compute(OpDescPtr, inputs, outputs)— Compute output tensors based on input tensors, for ConstantFoldingPass and DimensionComputePassCompute(NodePtr, outputs)— Compute output based on node information, some Kernel useCompute(NodePtr)— Only modify node attributes, for DimensionAdjustPassRegistered Host Kernel distributed by category:CategoryDirectoryOperator ListArray operationshost_kernels/array_ops/Reshape, Squeeze, SqueezeV3, Unsqueeze, UnsqueezeV3, ExpandDims, Rank, Shape, ShapeN, Size, Identity, Empty, BroadcastArgs, BroadcastGradientArgs, GatherShapesElement-wise computationhost_kernels/elewise_calculation_ops/Add, Sub, Mul, FloorDiv, FloorMod, Maximum, Greater, Rsqrt, CastSelection operationshost_kernels/selection_ops/Slice, SliceD, StridedSlice, GatherV2, RangeTransform operationshost_kernels/transformation_ops/Transpose, Permute, Transdata, FlattenV2Concatenate splithost_kernels/split_combination_ops/ConcatV2, ConcatOffset, Pack, UnpackPadding operationshost_kernels/pad_ops/FillReduction operationshost_kernels/reduce_ops/ReduceProdData flowhost_kernels/data_flow_ops/DynamicStitchCustomhost_kernels/custom_ops/ReFormat, SsdPriorBoxEach Kernel registers toKernelFactorythroughREGISTER_COMPUTE_NODE_KERNELmacro, runtime lookup by operator type.4.5.2 AICPU Host CPU EngineAs GE built-in Kernel supplement, through AICPU engine execute operators. Located atruntime/v2/engine/aicpu/andruntime/v1/hybrid/node_executor/host_cpu/.Load path islibconstant_folding_ops.so, provided by OPP (Operator Package), contains wider operator implementations.compiler/engines/cpu_engine/cpu_engine/constant_folding_stub/constant_folding_ops_stub.cppis compilation stub library (no actual implementation), runtime replaced by real OPP library.HostCpuEngineresponsible for:Loadinglibconstant_folding_ops.sodynamic libraryCreating operator instance throughOpKernelRegistryCallingRunHostCpuKernelexecute computation4.6 Compilation Flow IntegrationConstant folding called in multiple stages of GE graph compilation, forming multiple-round iterative optimization:GraphPrepare::ComputeConstantShape (preprocessing stage) │ ├── ReplaceWithEmptyConstPass (need_foldfalse, only mark) ├── DimensionComputePass (need_foldfalse, only mark) ├── ConstantClipPass ├── ConstantFoldingPass └── InferValueRangePass GraphManager::OptimizeStage1_2 (optimization stage1) │ ├── ReplaceWithEmptyConstPass ├── DimensionComputePass ├── ConstantClipPass ├── ConstantFoldingPass └── DimensionAdjustPass GraphManager::OptimizeStage2 (optimization stage2, after subgraph merge) │ ├── ConstantFoldingPass ├── ReshapeRemovePass ├── CondRemovePass ├── AssignRemovePass └── DimensionAdjustPass GraphOptimizerBeforeAutofuse (before auto-fusion) │ └── ConstantFoldingPassMulti-stage calling design considerations:Preprocessing stageruns DimensionComputePass in marking mode, first complete dimension inference then execute standard foldingOptimization stage1is constant folding main battlefield, contains all Pass complete executionOptimization stage2does constant folding once more for merged subgraph graph, eliminate new constant computations introduced by subgraph mergingBefore auto-fusionexecutes constant folding once more, ensure fusion Pass faces most optimized graph structure4.7 Coordination with Other OptimizationsConstant folding has coordination relationships with multiple other optimization Passes:ConstantClipPass: Handles weight clipping before constant folding, inserted Min/Max nodes will be folded again by subsequent ConstantFoldingPassDeadCodeElimination: Orphaned constant nodes produced by constant folding will be cleared by dead code eliminationCommonSubexpressionElimination: Under scenario sharing same constant input, after constant folding common subexpression elimination can further deduplicateWeightCompressJudge: Weight compression judgment callsGraphOptimizeUtility::ConstantFoldingfirst to do constant folding for weight-related nodes before executionDeleteNoConstFoldingFusionPass: FE quantization Pass, deletesATTR_NO_NEED_CONSTANT_FOLDINGattribute onAscendWeightQuantoperator in fusion stage, enabling it to be constant folding processedDecomposeLargeConstPass: Large constant split Pass inherits_is_from_constant_foldingattribute to split new constant nodesAscIrLowerer: In IR Lowering stage, special handling for control edges from constant folding (allow remove to get more fusion opportunities)CtrlEdgeTransferPass: Control edge relationships formed after constant folding need to be cleared by control edge transfer Pass【免费下载链接】geGEGraph Engine是面向昇腾的图编译器和执行器提供了计算图优化、多流并行、内存复用和模型下沉等技术手段加速模型执行效率减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考