Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

When I run the following part of the code it generates a broken pipe error. How can I fix this issue? The error generally refers to code line which says agent[0].start(). Thank you.

def main():
   np.random.seed(RANDOM_SEED)
   assert len(PACKET_SIZE) == A_DIM
   # create result directory
   if not os.path.exists(SUMMARY_DIR):
       os.makedirs(SUMMARY_DIR)
   # inter-process communication queues
   net_params_queues = []
   exp_queues = []
   for i in range(NUM_AGENTS):
       net_params_queues.append(mp.Queue(1))
       exp_queues.append(mp.Queue(1))
   # create a coordinator and multiple agent processes
   # (note: threading is not desirable due to python GIL)
   coordinator = mp.Process(target=central_agent,
                            args=(net_params_queues, exp_queues))
   coordinator.start()
   all_cooked_time, all_cooked_bw, _ = load_trace.load_trace(TRAIN_TRACES)
   agents = []
   print(NUM_AGENTS)
   for i in range(NUM_AGENTS):
       agents.append(mp.Process(target=agent,
                                args=(i, all_cooked_time, all_cooked_bw,
                                      net_params_queues[i],
                                      exp_queues[i])))
   #print(agents)
   #for i in range(NUM_AGENTS):
   agents[0].start()
   # wait unit training is done
   coordinator.join()
if __name__ == '__main__':
   main()
The complete traceback is shown below. I can add the complete code of the program if it is very necessary. I have copied it from somewhere and I am trying to understand it.
BrokenPipeError                           Traceback (most recent call last)
<ipython-input-24-152dc3806edb> in <module>
    333 if __name__ == '__main__':
--> 334     main()
<ipython-input-24-152dc3806edb> in main()
    325     #print(agents)
    326     #for i in range(NUM_AGENTS):
--> 327     agents[0].start()
    329     # wait unit training is done
C:\ProgramData\Anaconda3\lib\multiprocessing\process.py in start(self)
    110                'daemonic processes are not allowed to have children'
    111         _cleanup()
--> 112         self._popen = self._Popen(self)
    113         self._sentinel = self._popen.sentinel
    114         # Avoid a refcycle if the target function holds an indirect
C:\ProgramData\Anaconda3\lib\multiprocessing\context.py in _Popen(process_obj)
    221     @staticmethod
    222     def _Popen(process_obj):
--> 223         return _default_context.get_context().Process._Popen(process_obj)
    225 class DefaultContext(BaseContext):
C:\ProgramData\Anaconda3\lib\multiprocessing\context.py in _Popen(process_obj)
    320         def _Popen(process_obj):
    321             from .popen_spawn_win32 import Popen
--> 322             return Popen(process_obj)
    324     class SpawnContext(BaseContext):
C:\ProgramData\Anaconda3\lib\multiprocessing\popen_spawn_win32.py in __init__(self, process_obj)
     63             try:
     64                 reduction.dump(prep_data, to_child)
---> 65                 reduction.dump(process_obj, to_child)
     66             finally:
     67                 set_spawning_popen(None)
C:\ProgramData\Anaconda3\lib\multiprocessing\reduction.py in dump(obj, file, protocol)
     58 def dump(obj, file, protocol=None):
     59     '''Replacement for pickle.dump() using ForkingPickler.'''
---> 60     ForkingPickler(file, protocol).dump(obj)
BrokenPipeError: [Errno 32] Broken pipe
                It probably depends on what happens in the agent function, which is not shown. So there is not enough context here to give a meaningful answer. Also, if you get an exception, please add the complete traceback to your question.
– Roland Smith
                Aug 23, 2019 at 13:19
                @RolandSmith thank you for your comment. I will add the complete traceback to the question
– Frank Moses
                Aug 23, 2019 at 13:29
From the traceback it is clear that you are using multiprocessing from IPython on ms-windows.
Unfortunately, that doesn't work very well. AFAICT, it has to do with how IPython runs code in a notebook in combination with how multiprocessing has to work on ms-windows.
Try saving your code in a file and run that script from the command-line instead of in IPython.
                Thank you for your answer. Can you guide me how can I do this on my computer. I have jupyter installed but I do not know how to open the python command prompt.
– Frank Moses
                Aug 23, 2019 at 13:59
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.