0


open_ai-maddpg_Multiagent-particle-envs环境搭建过程

一.利用Anaconda安装python3.6环境

1.安装

下载源:

利用Anaconda安装python3.6环境_anaconda python3.6-CSDN博客

安装过程:
windows下载安装anaconda3.6图文教程 - 简书 (jianshu.com)

Python3.6版本+anaconda+PyCharm环境配置,全网最详细_anacondapython3.6-CSDN博客

Anaconda介绍、安装及使用教程 - 知乎 (zhihu.com)

2.新建项目,新建 Conda 环境

在这里插入图片描述

3.conda环境创建失败

网络错误CondaHTTPError: HTTP 000 CONNECTION FAILED for url :

参考链接

解决Anaconda创建环境时出现:CondaHTTPError: HTTP 000 CONNECTION FAILED for url <https://repo.anaconda.com/pkgs-CSDN博客

https://www.cnblogs.com/tianlang25/p/12433025.html

(1)打开Anaconda prompt,执行“conda config --remove-key channels”命令,恢复Anaconda的源为默认。
(2)恢复Anaconda镜像为清华的。在Anaconda prompt上执行以下命令:

conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/msys2/

conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/

conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/

conda config --set show_channel_urls yes

注意:这里建议不要用https,改为http

(3)将 .condarc (D盘/用户/用户名)文件中的“- defaults”删除;https,改为http

在这里插入图片描述

一定要关闭VPN再创建环境,否则会出现问题,因为要从清华库下载环境配置

二.程序搭建

1.依次安装gym、tensorfolw…

pip install gym==0.10.5
pip install tensorflow==1.14.0
pip install pyglet==1.5.9

(1)问题1:

WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:833)'),)': /simple/gym/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:833)'),)': /simple/gym/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:833)'),)': /simple/gym/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:833)'),)': /simple/gym/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:833)'),)': /simple/gym/
Could not fetch URL https://pypi.org/simple/gym/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/gym/ (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:833)'),)) - skipping
ERROR: Could not find a version that satisfies the requirement gym==0.10.5 (from versions: none)
ERROR: No matching distribution found for gym==0.10.5
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:833)'),)) - skipping

运行这个的时候要把梯子彻底退出,否则连接不上,报错:pip install Could not fetch URL

(1)问题2:

在这里插入图片描述

安装tensorflow时,遇到这样的问题: Attempting uninstall: wrapt
Found existing installation: wrapt 1.10.11
ERROR: Cannot uninstall ‘wrapt’. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

解决办法:

pip install -U --ignore-installed wrapt enum34 simplejson netaddr

然后再运行

pip install tensorflow==1.14.0

2.安装openAI的Multiagent-particle-envs

(1)下载Multiagent-particle-envs

下载网址:openai/multiagent-particle-envsgithub.com

git clone https://github.com/openai/multiagent-particle-envs

问题:

Cloning into ‘multiagent-particle-envs’…
fatal: unable to access ‘https://github.com/openai/multiagent-particle-envs/’: Failed to connect to github.com port 443 after 21067 ms: Couldn’t connect to server

解决:需要挂梯子,多是两次就好了

在这里插入图片描述

(2)先cd到 multiagent-particle-envs-master 中,
再执行

pip install -e .

在这里插入图片描述

3.安装openAI的maddgp

下载Maddpg开源项目文件

下载网址:openai/multiagent-particle-envsgithub.com

进入“\maddpg-master”目录;安装

pip install -e .

在这里插入图片描述

测试

安装完成后,输入如下代码进行测试

cd experiments
python train.py --scenario simple

出现如下画面说明开始训练:
在这里插入图片描述

原文件中,未打开可视化效果,如果想要看到画面,可以更改“train”中的第32行,将其中的

“ default=False”改为“default=True”,如图所示

在这里插入图片描述

运行后,出现如下画面

在这里插入图片描述

即“python train.py --scenario +环境名”,其中环境名根据实际应用需求进行修改。

由于渲染过程,采用可视化过程,训练速度会大大降低,同时由于环境较之前更为复杂,所以在“parser.add_argument(“–display”, action=“store_true”, default=True)”时,运行之初,可能看不到画面,可维持默认设置为“Flase”。

也可以在训练完数据后,进行可视化,运行如下代码:

python train.py --scenario simple_tag --display

4.遇到的问题

(1)文件记录有问题的话会出现以下报错:

在这里插入图片描述

进行以下更改

在这里插入图片描述

195行 改为: 196行

200行 改为: 201行

(2)文件记录路径若出现以下问题:

在这里插入图片描述

在目录experiments里,手动创建一个learning_curves文件夹

在这里插入图片描述

(3)【报错】已安装numpy,pycharm仍然报错ModuleNotFoundError: No module named ‘numpy‘

解决:

项目解释器

在这里插入图片描述

点“+”

在这里插入图片描述

添加缺少的库。

(4)import报错

ModuleNotFoundError: No module named ‘maddpg.common’

程序找不到模块,添加路径没有用,直接将.\maddpg-master\maddpg文件复制到使用的python编译器下面的库目录D:\Anaconda3\envs\pythonProject10\Lib\site-packages中。问题解决。

(5)报错 ModuleNotFoundError: No module named ‘gym’

实际情况是已经安装了:pip install gym==0.10.5,但是还是报错,鼠标放在gym(from gym.envs.registration import register)上,点击安装gym,等待安装完成即可

image-20240717095347157

(6)报错ImportError: cannot import name ‘prng’

对代码进行修改。因为新版本的gym已经没有prng功能了

将下面的

import gym
from gym.spaces import prng             # prng 功能已被取消

更改为:(改D:\Anaconda3\envs\pythonProject10\Lib\site-packages里面的)

# from gym.spaces import prng  # 注释掉

# random_array = prng.np_random.rand(self.num_discrete_space)  # replace this with following one
random_array = np.random.RandomState().rand(self.num_discrete_space)

(7)报错maddpg TypeError: softmax() got an unexpected keyword argument ‘axis’

将D:\Anaconda3\envs\pythonProject10\Lib\site-packages\maddpg\common\tf_util.py中的

tf.nn.softmax(x,axis=axis)

改为

tf.nn.softmax(x,dim=axis)

可以避免冲突,从而运行程序。

(8)显示的时候报错ImportError: cannot import name ‘reraise’

外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传

进行如下添加更改

外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传

四.参考链接

1.参考

https://blog.csdn.net/wzc_123456/article/details/121655808(MADDPG——环境搭建(多智能体强化学习))

论文讲解(基于Multi-Agent的无人机集群体系自主作战系统设计)

https://blog.csdn.net/longtengzhangjie/article/details/138111316

2.问题总结参考链接:

https://blog.csdn.net/Fearless_Sun/article/details/132717061

3.Multiagent-particle-envs环境的介绍可以阅读:

https://blog.csdn.net/weixin_43483381/article/details/114240352

外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传

OpenAI 多智能体强化学习环境(multiagent-particle-envs)详解 - gingkg - 博客园 (cnblogs.com)

4.对于Multiagent-particle-envs的代码解析可以阅读:

https://zhuanlan.zhihu.com/p/555515701

(多智能体环境MPE(multiagent particle environment)介绍与代码分析)

代码分析Xmind流程图

5.论文学习

https://aistudio.baidu.com/projectdetail/637951(从代码到论文理解并复现MADDPG算法(PARL))

https://jianengli.github.io/2021/03/19/rl/maddpg/(maddpg原理以及代码解读)

https://hrl.boyuai.com/chapter/3/%E5%A4%9A%E6%99%BA%E8%83%BD%E4%BD%93%E5%BC%BA%E5%8C%96%E5%AD%A6%E4%B9%A0%E8%BF%9B%E9%98%B6/(动手学强化学习)

Hands-on-RL/rl_utils.py at main · boyu-ai/Hands-on-RL · GitHub(动手学强化学习)的代码

https://nqtr.cn/news/show-4688807.html?action=onClick(环境改动笔记)

https://github.com/shariqiqbal2810(pytorch maddpg代码)

https://m.fx361.com/news/2023/0519/22338372.html (项目场景描述)

https://oss.wanfangdata.com.cn/www/%E5%9F%BA%E4%BA%8E%E5%BC%BA%E5%8C%96%E5%AD%A6%E4%B9%A0%E7%9A%84%E5%A4%9A%E6%99%BA%E8%83%BD%E4%BD%93%E5%8D%8F%E5%90%8C%E8%B7%AF%E5%BE%84%E8%A7%84%E5%88%92%E4%B8%8E%E7%BC%96%E9%98%9F%E5%8C%85%E5%9B%B4.ashx?isread=true&type=degree&resourceId=D02625563&transaction=%7B%22id%22%3Anull%2C%22transferOutAccountsStatus%22%3Anull%2C%22transaction%22%3A%7B%22id%22%3A%221816437661934661632%22%2C%22status%22%3A1%2C%22createDateTime%22%3Anull%2C%22payDateTime%22%3A1721907461042%2C%22authToken%22%3A%22TGT-377980-ZchyqbNrNylZGhFOdog5FepA4whyXHVZV3x6NN0bFSTeyOfLzN-auth-iploginservice-6bf5fd78c6-2b9nh%22%2C%22user%22%3A%7B%22accountType%22%3A%22Group%22%2C%22key%22%3A%22hebgcdx%22%7D%2C%22transferIn%22%3A%7B%22accountType%22%3A%22Income%22%2C%22key%22%3A%22ThesisFulltext%22%7D%2C%22transferOut%22%3A%7B%22GTimeLimit.hebgcdx%22%3A30.0%7D%2C%22turnover%22%3A30.0%2C%22orderTurnover%22%3A30.0%2C%22productDetail%22%3A%22degree_D02625563%22%2C%22productTitle%22%3Anull%2C%22userIP%22%3A%22222.27.246.1%22%2C%22organName%22%3Anull%2C%22memo%22%3Anull%2C%22orderUser%22%3A%22hebgcdx%22%2C%22orderChannel%22%3A%22pc%22%2C%22payTag%22%3A%22%22%2C%22webTransactionRequest%22%3Anull%2C%22signature%22%3A%22Gog3p8LhNU%2BNlHakx0s06%2Fl%2Bap9zdMtOzg6VAANrSecyR0vtUlHYUpQs1BJIBT6rxh7PuyDcmuh8%5CnA8oKR0%2FaSpwOu1UkxNiu%2FnptDnH4fERrt92HcUVPgbvmXyJVrGVH9bBRbWPGX%2Bw1Cr0FwrLG2Jo%2B%5CnjderTHn8wIAI5eJl%2FN0%3D%22%7D%2C%22isCache%22%3Afalse%7D


本文转载自: https://blog.csdn.net/qq_43159983/article/details/140473542
版权归原作者 榆次小青年 所有, 如有侵权,请联系我们删除。

“open_ai-maddpg_Multiagent-particle-envs环境搭建过程”的评论:

还没有评论