是否可以以某种方式自定义 QTabWidget？

Question

DarMaster

Asked:2022-04-07 23:07:17 +0000 UTC2022-04-07 23:07:17 +0000 UTC 2022-04-07 23:07:17 +0000 UTC

如何在图像解析代码中添加多处理

772

学习了如何使用多线程，但无法附加多处理模块。

编码：

import requests
import multiprocessing
from bs4 import BeautifulSoup 

storage_number = 100
image_num = 0

link = 'https://wallpaperscraft.ru/catalog/art/1920x1080'
link_image = 'https://images.wallpaperscraft.ru/image'

for i in range(1,storage_number):
    responce = requests.get(f'{link}/page{i}').text
    soup = BeautifulSoup(responce, 'lxml')
    block = soup.find('ul',class_ = 'wallpapers__list')
    all_image = block.find_all('li', class_ = 'wallpapers__item')


    for image in all_image:
        aimage_href = image.find('a').get('href')
        url_image = link_image + aimage_href[9:-10] + '_1920x1080.jpg' 
        image_bytes = requests.get(url_image).content
        with open(f'image/{image_num}.jpg', 'wb') as file:
            file.write(image_bytes)
        image_num += 1

试图：

import requests
from multiprocessing import Pool
from bs4 import BeautifulSoup 



def multi_img():
    storage_number = 2
    image_num = 0
    link       = 'https://wallpaperscraft.ru/catalog/art/1920x1080'
    link_image = 'https://images.wallpaperscraft.ru/image'

    for i in range(1,storage_number):
        responce = requests.get(f'{link}/page{i}').text
        soup = BeautifulSoup(responce, 'lxml')
        block = soup.find('ul',class_ = 'wallpapers__list')
        all_image = block.find_all('li', class_ = 'wallpapers__item')


        for image in all_image:
            aimage_href = image.find('a').get('href')
            url_image = link_image + aimage_href[9:-10] + '_1920x1080.jpg' 
            image_bytes = requests.get(url_image).content
            with open(f'image/{image_num}.jpg', 'wb') as file:
                file.write(image_bytes)
            image_num += 1

with Pool(5) as p:
    multi_img()

给我：

  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 268, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "c:\Python_project\Request\foto_wall_new.py", line 28, in <module>
    with Pool(5) as p:
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\context.py", line 119, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,   
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\pool.py", line 212, in __init__
    self._repopulate_pool()
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\pool.py", line 303, in _repopulate_pool
    return self._repopulate_pool_static(self._ctx, self.Process,      
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\pool.py", line 326, in _repopulate_pool_static
    w.start()
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\popen_spawn_win32.py", line 45, in __init__
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 154, in get_preparation_data
    _check_not_importing_main()
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 134, in _check_not_importing_main      
    raise RuntimeError('''
RuntimeError:
        An attempt has been made to start a new process before the    
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your 
        child processes and you have forgotten to use the proper idiom        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program     
        is not going to be frozen to produce an executable.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 125, in _main
    prepare(preparation_data)
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 268, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "c:\Python_project\Request\foto_wall_new.py", line 28, in <module>
    with Pool(5) as p:
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\context.py", line 119, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,   
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\pool.py", line 212, in __init__
    self._repopulate_pool()
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\pool.py", line 303, in _repopulate_pool
    return self._repopulate_pool_static(self._ctx, self.Process,      
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\pool.py", line 326, in _repopulate_pool_static
    w.start()
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\popen_spawn_win32.py", line 45, in __init__
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 154, in get_preparation_data
    _check_not_importing_main()
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 134, in _check_not_importing_main      
    raise RuntimeError('''
RuntimeError:
        An attempt has been made to start a new process before the    
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your 
        child processes and you have forgotten to use the proper idiom        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program     
        is not going to be frozen to produce an executable.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 116, in spawn_main
Traceback (most recent call last):
  File "<string>", line 1, in <module>
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 125, in _main
    prepare(preparation_data)
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 116, in spawn_main
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 236, in prepare
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 125, in _main
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
    prepare(preparation_data)
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 236, in prepare
    main_content = runpy.run_path(main_path,
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 268, in run_path
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
    return _run_module_code(code, init_globals, run_name,
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 97, in _run_module_code
    main_content = runpy.run_path(main_path,
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 268, in run_path
    _run_code(code, mod_globals, init_globals,
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 87, in _run_code
    return _run_module_code(code, init_globals, run_name,
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 97, in _run_module_code
    exec(code, run_globals)
  File "c:\Python_project\Request\foto_wall_new.py", line 28, in <module>
    _run_code(code, mod_globals, init_globals,
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 87, in _run_code
    with Pool(5) as p:
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\context.py", line 119, in Pool
    exec(code, run_globals)
  File "c:\Python_project\Request\foto_wall_new.py", line 28, in <module>
    return Pool(processes, initializer, initargs, maxtasksperchild,   
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\pool.py", line 212, in __init__
    with Pool(5) as p:
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\context.py", line 119, in Pool
    self._repopulate_pool()
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\pool.py", line 303, in _repopulate_pool
    return Pool(processes, initializer, initargs, maxtasksperchild,   
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\pool.py", line 212, in __init__
    return self._repopulate_pool_static(self._ctx, self.Process,
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\pool.py", line 326, in _repopulate_pool_static
    self._repopulate_pool()
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\pool.py", line 303, in _repopulate_pool
    w.start()
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\process.py", line 121, in start
    return self._repopulate_pool_static(self._ctx, self.Process,      
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\pool.py", line 326, in _repopulate_pool_static
    self._popen = self._Popen(self)
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\context.py", line 327, in _Popen
    w.start()
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\process.py", line 121, in start
    return Popen(process_obj)
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\popen_spawn_win32.py", line 45, in __init__
    self._popen = self._Popen(self)
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\context.py", line 327, in _Popen
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 154, in get_preparation_data
    return Popen(process_obj)
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\popen_spawn_win32.py", line 45, in __init__
    _check_not_importing_main()
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 134, in _check_not_importing_main      
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 154, in get_preparation_data
    raise RuntimeError('''
RuntimeError:
        An attempt has been made to start a new process before the    
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your 
        child processes and you have forgotten to use the proper idiom        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program     
        is not going to be frozen to produce an executable.    _check_not_importing_main()

  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 134, in _check_not_importing_main      
    raise RuntimeError('''
RuntimeError:
        An attempt has been made to start a new process before the    
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your 
        child processes and you have forgotten to use the proper idiom        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program     
        is not going to be frozen to produce an executable.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 125, in _main
    prepare(preparation_data)
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "C:\Users\Владелец\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 268, in run_path
PS C:\Python_project>

虽然有效，但加速度为0

1 个回答

Voted

CrazyElf · Answer 1 · 2022-04-08T00:41:21Z

Best Answer

CrazyElf

2022-04-08T00:41:21Z2022-04-08T00:41:21Z

好吧，在同一个地方，错误说您需要在构造中构建池调用：

if __name__ == '__main__':

而且，此外，您调用相同的函数而不使用参数，根本不使用创建的池。一般来说，你需要这样的东西：

storage_number = 100
...

def multi_img(i):
   ...
   # for выпилить из функции, i будет приходить как параметр

if __name__ == '__main__':
    with Pool(5) as p:
        p.map(multi_img, range(1,storage_number))

2

如何在图像解析代码中添加多处理

表格填充不起作用

提示 50/50，有两个，其中一个是正确的

在 PyQt5 中停止进程

我的脚本不工作

在文本文件中写入和读取列表

如何像屏幕截图中那样并排排列这些块？

确定文本文件中每一行的字符数

将接口对象传递给 JAVA 构造函数

正确更新数据库中的数据

Python解析不是css

如何在图像解析代码中添加多处理

1 个回答

相关问题