minivisor

Overview

Minivisor是用python开发的一套通用的分布式爬虫进程管理程序，实时监测进程的状态，保证在服务器上启动一定数量的爬虫进程，并在其异常退出时自动重启。Minivisor is a general-purpose distributed crawler process management program developed in Python, which monitors the status of processes in real time, ensures that a certain number of crawler processes are started on the server, and restarts automatically when it exits abnormally. Open-source project by wuwenrufeng, published on GitHub.

Highlights

• 1 star on GitHub
• 1 fork
• Primary language: Python
• Open source — view the code and contribute on GitHub

Built with

• Python
• Processing
• Spider
• Task Manager