主页 / 问题 / 1591374
Accepted
polsok
Asked:2024-08-21 00:41:53 +0000 UTC2024-08-21 00:41:53 +0000 UTC

将yaml文件解析成表

  • 772

有几个yaml文件：

文件1

 phone:
   os: android
   size:
     length: 80
     height: 10
     weight: 40
   model: SE-35
   price: 20000

文件2

 phone:
   os: apple
   model: Iphone 15
   price: 150000
   color: gold

文件3

 phone:
   os: apple
   model: Iphone 15
   color: black
   exist: absent

文件4

 phone:
   os: android
   size:
     length: 75
     height: 8
     weight: 36
   camera:
     front: 4 mp
     back: 64 mp
       zoom: 3
   model: SE-35
   price: 20000

它们需要以表格形式呈现，其中第一列是键，其余是值。

该表将如下所示：

电话 文件1 文件2 文件3 文件4
电话操作系统 安卓 苹果 苹果 安卓
手机.尺寸.长度 80 75
手机.尺寸.高度 10 8
手机.尺寸.重量 40 35
手机型号 SE-35 iPhone 15 iPhone 15
电话.价格 20000 150000 20000
电话颜色 金子 黑色的
电话.存在 缺席的
手机.相机.前置 4 议员
电话.相机.返回 64 点
手机.相机.后退.缩放 3

其实这样的文件还有很多，我只是举了一些例子

我试图解析它们：这是代码：

import os
import yaml
import pandas as pd

# получаем данные из файла
def GetData(data, prefix):
  if isinstance(data, dict):
    for k, v in data.items():
        yield from GetData(v, f'{prefix}/{k}')
  else:
    yield (prefix, data)

# находим все файлы в директории
def findYamlFiles(directory):
  for root, dirs, files in os.walk(directory):
    for file in files:
        if file.endswith('.yaml' or '.yml'):
            with open(os.path.join(root, file), 'r') as f:
              yaml_data = yaml.safe_load(f)
              df = pd.DataFrame(GetData(yaml_data, ''), columns=['phone', 'rezults'])

path = "~/phones"
findYamlFiles(path)

总的来说，在 df 中我得到了一个完全解析的 yaml 文件，由两列组成，但我无法将所有 df 收集到一张表中。

python

3 个回答

  1. 如果全部df具有相同的结构：

    def findYamlFiles(directory):
    main_df = None
    all_df = []
    for root, dirs, files in os.walk(directory):
        for file in files:
            if file.endswith('.yaml' or '.yml'):
                with open(os.path.join(root, file), 'r') as f:
                    yaml_data = yaml.safe_load(f)
                    all_df.append(pd.DataFrame(GetData(yaml_data, ''), columns=['Column 1', 'Column 2']))
    • 1 个选项
    print(pd.concat(all_df, ignore_index=True))

    结论：

                  Column 1 Column 2
0            /phone/os  android
1   /phone/size/length       80
2   /phone/size/height       10
3   /phone/size/weight       40
4         /phone/model    SE-35
5         /phone/price    20000
6            /phone/os  android
7   /phone/size/length       80
8   /phone/size/height       10
9   /phone/size/weight       40
10        /phone/model    SE-35
11        /phone/price    20000
    • 选项2
    print(pd.concat(all_df, axis=1))

    结论：

                 Column 1 Column 2            Column 1 Column 2
0           /phone/os  android           /phone/os  android
1  /phone/size/length       80  /phone/size/length       80
2  /phone/size/height       10  /phone/size/height       10
3  /phone/size/weight       40  /phone/size/weight       40
4        /phone/model    SE-35        /phone/model    SE-35
5        /phone/price    20000        /phone/price    20000
    • 选项3
    print(pd.merge(*all_df, on='Column 1'))

    结论：

                 Column 1 Column 2_x Column 2_y
0           /phone/os    android    android
1  /phone/size/length         80         80
2  /phone/size/height         10         10
3  /phone/size/weight         40         40
4        /phone/model      SE-35      SE-35
5        /phone/price      20000      20000
    • 4个选项（超过2个文件）
    df = pd.DataFrame(GetData(yaml_data, ''), columns=['Column 1', file])
if main_df is None:
    main_df = df
    continue
main_df = pd.merge(main_df, df, on="Column 1")

    结论：

                 Column 1 car.yaml car2.yaml car3.yaml
0           /phone/os  android   android   android
1  /phone/size/length       80        80        80
2  /phone/size/height       10        10        10
3  /phone/size/weight       40        40        40
4        /phone/model    SE-35     SE-35     SE-35
5        /phone/price    20000     20000     20000

    pd.合并

    pd.concat

    • 2
  2. Best Answer

    一般来说，您可能可以只使用json_normalize：

    files = ["f1.yaml", "f2.yaml", "f3.yaml", "f4.yaml"]

def framify(filename: str):
    with open(filename, "r") as f:
        retval = pd.DataFrame(pd.json_normalize(yaml.safe_load(f))).T
        retval = retval.rename(columns={0:filename})
    return retval
    
res = pd.concat([framify(file) for file in files], axis=1)

    资源：

                        f1.yaml    f2.yaml    f3.yaml  f4.yaml
phone.os            android      apple      apple  android
phone.size.length        80        NaN        NaN       75
phone.size.height        10        NaN        NaN        8
phone.size.weight        40        NaN        NaN       36
phone.model           SE-35  Iphone 15  Iphone 15    SE-35
phone.price           20000     150000        NaN    20000
phone.color             NaN       gold      black      NaN
phone.exist             NaN        NaN     absent      NaN
phone.camera.front      NaN        NaN        NaN     4 mp
phone.camera.back       NaN        NaN        NaN    64 mp
phone.camera.zoom       NaN        NaN        NaN        3
    • 2

  3. 您可以以不同的方式构建数据：

    import os
import yaml
import pandas as pd
from rich import print, inspect

# Укажите путь к директории с YAML-файлами
directory_path = r'C:\YAMLs'

# Словарь для хранения данных
data = {}

# Проходим по всем файлам в директории
for filename in os.listdir(directory_path):
    if filename.endswith('.yaml') or filename.endswith('.yml'):
        file_path = os.path.join(directory_path, filename)

        # Читаем YAML-файл
        with open(file_path, 'r', encoding='utf-8') as file:
            yaml_content = yaml.safe_load(file)
            phone_data = yaml_content['phone']
            # Добавляем данные в словарь
            data[filename] = {
                'os': phone_data['os'],
                'size.length': phone_data['size']['length'],
                'size.height': phone_data['size']['height'],
                'size.weight': phone_data['size']['weight'],
                'model': phone_data['model'],
                'price': phone_data['price']
            }

# Создаем DataFrame
df = pd.DataFrame(data).T

# Устанавливаем многоуровневый индекс для столбцов
df.columns = pd.MultiIndex.from_product([['phone'], df.columns])

# Печатаем результат
inspect(df, value=True)

    输出：

    ╭───────────────── <class 'pandas.core.frame.DataFrame'> ──────────────────╮
│ Two-dimensional, size-mutable, potentially heterogeneous tabular data.   │
│                                                                          │
│ ╭──────────────────────────────────────────────────────────────────────╮ │
│ │ │   │   │    phone                                                   │ │
│ │ │   │   │   │   os size.length size.height size.weight  model  price │ │
│ │ pho.yaml       ios          90          10          50  fE-35  70000 │ │
│ │ pho2.yml   android          80          12          40  SE-35  20000 │ │
│ │ pho3.yaml  windows         100          10          50  QE-95  37000 │ │
│ ╰──────────────────────────────────────────────────────────────────────╯ │
~                                                                          ~
╰──────────────────────────────────────────────────────────────────────────╯
    • 1

相关问题