工业领域的产品手册多以PDF形式发布参数分散在不同品牌的样本册中格式各异。本文记录一套用Python从PDF提取表格、标准化字段、清洗参数值的完整方案供同类数据处理场景参考。一、问题与方案PDF中的参数表多为矢量文本可直接提取无需OCR。技术栈如下表格工具 用途pdfplumber PDF表格提取pandas 数据清洗re 正则标准化二、PDF表格提取Pythonimport pdfplumberimport pandas as pdimport osdef extract_tables_from_pdf(pdf_path):all_tables []with pdfplumber.open(pdf_path) as pdf:for page_num, page in enumerate(pdf.pages, 1):tables page.extract_tables()for table in tables:if len(table) 1:df pd.DataFrame(table[1:], columnstable[0])df[‘source_page’] page_numdf[‘source_file’] os.path.basename(pdf_path)all_tables.append(df)return pd.concat(all_tables, ignore_indexTrue) if all_tables else pd.DataFrame()使用示例df_raw extract_tables_from_pdf(“catalog_a.pdf”)print(f提取完成共 {len(df_raw)} 行)print(df_raw.head())三、字段标准化不同厂商的列名不统一建立映射表统一字段PythonFIELD_MAP {‘型号’: ‘model’, ‘Model’: ‘model’, ‘型式’: ‘model’,‘检测距离’: ‘detection_distance’, ‘检测范围’: ‘detection_distance’, ‘Range’: ‘detection_distance’,‘输出类型’: ‘output_type’, ‘输出’: ‘output_type’, ‘Output’: ‘output_type’,‘响应时间’: ‘response_time’, ‘响应频率’: ‘response_time’,‘电源电压’: ‘supply_voltage’, ‘工作电压’: ‘supply_voltage’,‘防护等级’: ‘protection_rating’, ‘IP rating’: ‘protection_rating’}def normalize_columns(df):return df.rename(columns{k: v for k, v in FIELD_MAP.items() if k in df.columns})df normalize_columns(df_raw)四、参数值清洗工业参数的单位与格式混乱需统一Pythonimport redef clean_value(value, field):if pd.isna(value) or value ‘’:return Nonev str(value).strip()if field detection_distance: m re.search(r(\d(?:\.\d)?)\s*(mm|cm|m), v) return f{m.group(1)}{m.group(2)} if m else v elif field response_time: m re.search(r(\d(?:\.\d)?)\s*(ms|μs|us), v) if m: num, unit m.groups() return f{float(num)/1000:.3f}ms if unit in [μs, us] else f{num}ms return v elif field supply_voltage: m re.search(r(\d)[Vv]?~?(\d)?[Vv]?, v) if m: low, high m.groups() return f{low}V~{high}V if high else f{low}V return v return v批量清洗for field in [‘detection_distance’, ‘response_time’, ‘supply_voltage’]:if field in df.columns:df[field] df[field].apply(lambda x: clean_value(x, field))五、批量处理多文件Pythonimport globdef batch_process(input_dir, output_file):all_data []for pdf in glob.glob(os.path.join(input_dir, “*.pdf”)):df extract_tables_from_pdf(pdf)if not df.empty:df normalize_columns(df)for field in [‘detection_distance’, ‘response_time’, ‘supply_voltage’]:if field in df.columns:df[field] df[field].apply(lambda x: clean_value(x, field))all_data.append(df)if all_data: result pd.concat(all_data, ignore_indexTrue) result.to_csv(output_file, indexFalse, encodingutf-8-sig) print(f完成共处理 {len(all_data)} 个文件输出 {len(result)} 行) return result return pd.DataFrame()执行df_final batch_process(“./pdf_catalogs/”, “output.csv”)六、替代型号匹配示例基于清洗后的数据按核心参数匹配Pythondef find_matches(target, database):candidates database[(database[‘output_type’] target[‘output_type’]) (database[‘detection_distance’].str.contains(target[‘detection_distance’].replace(‘mm’,‘’), naFalse))]return candidates[[‘model’, ‘detection_distance’, ‘output_type’, ‘supply_voltage’]]七、常见问题表格问题 原因 解决表格跨页断行 PDF分页导致 按型号分组向下填充缺失值单位不统一 各厂习惯不同 清洗函数统一换算页眉页脚混入 被识别为表格 过滤行数小于3的表八、输出示例清洗后的标准格式JSON{“model”: “FG-18N”,“detection_distance”: “500mm”,“output_type”: “NPN”,“response_time”: “0.500ms”,“supply_voltage”: “12V~24V”,“protection_rating”: “IP67”}九、扩展方向接入向量数据库实现语义检索接入LLM做自然语言选型问答构建Web查询界面供前端调用