当前位置：首页>学习笔记>强化学习笔记(3)——策略优化

强化学习笔记(3)——策略优化

  • 2026-03-02 17:06:44
强化学习笔记(3)——策略优化

强化学习笔记(3)——策略优化

文章主要介绍RL中策略优化算法。
在环境状态下，策略产生行为，行为产生轨迹，轨迹在环境中获得奖励。
获得最佳奖励的过程，就是产生最佳动作序列的过程，也就是设计最佳策略参数使得最佳动作序列输出的概率最大的过程。
这个过程中有一些工程技巧，比如策略梯度对数求导、在策略梯度中增加优势方程降低梯度方差、采用reward-to-go的奖励形式。#policygradient算法 #深度学习笔记 #策略优化笔记

思维之树

微信扫一扫赞赏作者喜欢作者

    正在加载...
      正在加载...
      系统错误，请稍后重试
      名称已清空
      微信扫一扫赞赏作者
      喜欢作者其它金额
      文章
      暂无文章
      喜欢作者
      其它金额
      最低赞赏 ¥0
      其它金额
      赞赏金额
      ¥
      最低赞赏 ¥0
      1
      2
      3
      4
      5
      6
      7
      8
      9
      0
      .
      广东,31分钟前,2026年2月28日
      已修改
      本文来自网友投稿或网络内容，如有侵犯您的权益请联系我们删除，联系邮箱：wyl860211@qq.com 。

      【二造最全合集】2025年二级造价工程师学习备考资料分享(各个省份)【最新超全】收藏这篇就好了!

      【学习资料】一图读懂丨民主评议党员应知应会

      最新文章

      随机文章

      基本 文件 流程 错误 SQL 调试
      1. 请求信息 : 2026-03-03 00:03:21 HTTP/2.0 GET : https://67808.cn/a/471251.html
      2. 运行时间 : 0.205481s [ 吞吐率：4.87req/s ] 内存消耗：4,837.38kb 文件加载：140
      3. 缓存信息 : 0 reads,0 writes
      4. 会话信息 : SESSION_ID=d6bb71977b6beabe765515d41a8e4463
      1. /yingpanguazai/ssd/ssd1/www/no.67808.cn/public/index.php ( 0.79 KB )
      2. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/autoload.php ( 0.17 KB )
      3. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/composer/autoload_real.php ( 2.49 KB )
      4. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/composer/platform_check.php ( 0.90 KB )
      5. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/composer/ClassLoader.php ( 14.03 KB )
      6. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/composer/autoload_static.php ( 4.90 KB )
      7. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-helper/src/helper.php ( 8.34 KB )
      8. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-validate/src/helper.php ( 2.19 KB )
      9. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/helper.php ( 1.47 KB )
      10. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/stubs/load_stubs.php ( 0.16 KB )
      11. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/Exception.php ( 1.69 KB )
      12. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-container/src/Facade.php ( 2.71 KB )
      13. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/symfony/deprecation-contracts/function.php ( 0.99 KB )
      14. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/symfony/polyfill-mbstring/bootstrap.php ( 8.26 KB )
      15. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/symfony/polyfill-mbstring/bootstrap80.php ( 9.78 KB )
      16. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/symfony/var-dumper/Resources/functions/dump.php ( 1.49 KB )
      17. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-dumper/src/helper.php ( 0.18 KB )
      18. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/symfony/var-dumper/VarDumper.php ( 4.30 KB )
      19. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/App.php ( 15.30 KB )
      20. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-container/src/Container.php ( 15.76 KB )
      21. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/psr/container/src/ContainerInterface.php ( 1.02 KB )
      22. /yingpanguazai/ssd/ssd1/www/no.67808.cn/app/provider.php ( 0.19 KB )
      23. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/Http.php ( 6.04 KB )
      24. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-helper/src/helper/Str.php ( 7.29 KB )
      25. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/Env.php ( 4.68 KB )
      26. /yingpanguazai/ssd/ssd1/www/no.67808.cn/app/common.php ( 0.03 KB )
      27. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/helper.php ( 18.78 KB )
      28. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/Config.php ( 5.54 KB )
      29. /yingpanguazai/ssd/ssd1/www/no.67808.cn/config/app.php ( 0.95 KB )
      30. /yingpanguazai/ssd/ssd1/www/no.67808.cn/config/cache.php ( 0.78 KB )
      31. /yingpanguazai/ssd/ssd1/www/no.67808.cn/config/console.php ( 0.23 KB )
      32. /yingpanguazai/ssd/ssd1/www/no.67808.cn/config/cookie.php ( 0.56 KB )
      33. /yingpanguazai/ssd/ssd1/www/no.67808.cn/config/database.php ( 2.48 KB )
      34. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/facade/Env.php ( 1.67 KB )
      35. /yingpanguazai/ssd/ssd1/www/no.67808.cn/config/filesystem.php ( 0.61 KB )
      36. /yingpanguazai/ssd/ssd1/www/no.67808.cn/config/lang.php ( 0.91 KB )
      37. /yingpanguazai/ssd/ssd1/www/no.67808.cn/config/log.php ( 1.35 KB )
      38. /yingpanguazai/ssd/ssd1/www/no.67808.cn/config/middleware.php ( 0.19 KB )
      39. /yingpanguazai/ssd/ssd1/www/no.67808.cn/config/route.php ( 1.89 KB )
      40. /yingpanguazai/ssd/ssd1/www/no.67808.cn/config/session.php ( 0.57 KB )
      41. /yingpanguazai/ssd/ssd1/www/no.67808.cn/config/trace.php ( 0.34 KB )
      42. /yingpanguazai/ssd/ssd1/www/no.67808.cn/config/view.php ( 0.82 KB )
      43. /yingpanguazai/ssd/ssd1/www/no.67808.cn/app/event.php ( 0.25 KB )
      44. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/Event.php ( 7.67 KB )
      45. /yingpanguazai/ssd/ssd1/www/no.67808.cn/app/service.php ( 0.13 KB )
      46. /yingpanguazai/ssd/ssd1/www/no.67808.cn/app/AppService.php ( 0.26 KB )
      47. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/Service.php ( 1.64 KB )
      48. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/Lang.php ( 7.35 KB )
      49. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/lang/zh-cn.php ( 13.70 KB )
      50. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/initializer/Error.php ( 3.31 KB )
      51. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/initializer/RegisterService.php ( 1.33 KB )
      52. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/services.php ( 0.14 KB )
      53. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/service/PaginatorService.php ( 1.52 KB )
      54. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/service/ValidateService.php ( 0.99 KB )
      55. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/service/ModelService.php ( 2.04 KB )
      56. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-trace/src/Service.php ( 0.77 KB )
      57. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/Middleware.php ( 6.72 KB )
      58. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/initializer/BootService.php ( 0.77 KB )
      59. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/Paginator.php ( 11.86 KB )
      60. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-validate/src/Validate.php ( 63.20 KB )
      61. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/Model.php ( 23.55 KB )
      62. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/model/concern/Attribute.php ( 21.05 KB )
      63. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/model/concern/AutoWriteData.php ( 4.21 KB )
      64. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/model/concern/Conversion.php ( 6.44 KB )
      65. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/model/concern/DbConnect.php ( 5.16 KB )
      66. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/model/concern/ModelEvent.php ( 2.33 KB )
      67. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/model/concern/RelationShip.php ( 28.29 KB )
      68. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-helper/src/contract/Arrayable.php ( 0.09 KB )
      69. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-helper/src/contract/Jsonable.php ( 0.13 KB )
      70. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/model/contract/Modelable.php ( 0.09 KB )
      71. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/Db.php ( 2.88 KB )
      72. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/DbManager.php ( 8.52 KB )
      73. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/Log.php ( 6.28 KB )
      74. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/Manager.php ( 3.92 KB )
      75. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/psr/log/src/LoggerTrait.php ( 2.69 KB )
      76. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/psr/log/src/LoggerInterface.php ( 2.71 KB )
      77. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/Cache.php ( 4.92 KB )
      78. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/psr/simple-cache/src/CacheInterface.php ( 4.71 KB )
      79. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-helper/src/helper/Arr.php ( 16.63 KB )
      80. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/cache/driver/File.php ( 7.84 KB )
      81. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/cache/Driver.php ( 9.03 KB )
      82. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/contract/CacheHandlerInterface.php ( 1.99 KB )
      83. /yingpanguazai/ssd/ssd1/www/no.67808.cn/app/Request.php ( 0.09 KB )
      84. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/Request.php ( 55.78 KB )
      85. /yingpanguazai/ssd/ssd1/www/no.67808.cn/app/middleware.php ( 0.25 KB )
      86. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/Pipeline.php ( 2.61 KB )
      87. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-trace/src/TraceDebug.php ( 3.40 KB )
      88. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/middleware/SessionInit.php ( 1.94 KB )
      89. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/Session.php ( 1.80 KB )
      90. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/session/driver/File.php ( 6.27 KB )
      91. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/contract/SessionHandlerInterface.php ( 0.87 KB )
      92. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/session/Store.php ( 7.12 KB )
      93. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/Route.php ( 23.73 KB )
      94. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/route/RuleName.php ( 5.75 KB )
      95. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/route/Domain.php ( 2.53 KB )
      96. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/route/RuleGroup.php ( 22.43 KB )
      97. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/route/Rule.php ( 26.95 KB )
      98. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/route/RuleItem.php ( 9.78 KB )
      99. /yingpanguazai/ssd/ssd1/www/no.67808.cn/route/app.php ( 1.72 KB )
      100. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/facade/Route.php ( 4.70 KB )
      101. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/route/dispatch/Controller.php ( 4.74 KB )
      102. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/route/Dispatch.php ( 10.44 KB )
      103. /yingpanguazai/ssd/ssd1/www/no.67808.cn/app/controller/Index.php ( 4.81 KB )
      104. /yingpanguazai/ssd/ssd1/www/no.67808.cn/app/BaseController.php ( 2.05 KB )
      105. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/facade/Db.php ( 0.93 KB )
      106. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/db/connector/Mysql.php ( 5.44 KB )
      107. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/db/PDOConnection.php ( 52.47 KB )
      108. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/db/Connection.php ( 8.39 KB )
      109. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/db/ConnectionInterface.php ( 4.57 KB )
      110. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/db/builder/Mysql.php ( 16.58 KB )
      111. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/db/Builder.php ( 24.06 KB )
      112. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/db/BaseBuilder.php ( 27.50 KB )
      113. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/db/Query.php ( 15.71 KB )
      114. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/db/BaseQuery.php ( 45.13 KB )
      115. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/db/concern/TimeFieldQuery.php ( 7.43 KB )
      116. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/db/concern/AggregateQuery.php ( 3.26 KB )
      117. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/db/concern/ModelRelationQuery.php ( 20.07 KB )
      118. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/db/concern/ParamsBind.php ( 3.66 KB )
      119. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/db/concern/ResultOperation.php ( 7.01 KB )
      120. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/db/concern/WhereQuery.php ( 19.37 KB )
      121. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/db/concern/JoinAndViewQuery.php ( 7.11 KB )
      122. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/db/concern/TableFieldInfo.php ( 2.63 KB )
      123. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-orm/src/db/concern/Transaction.php ( 2.77 KB )
      124. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/log/driver/File.php ( 5.96 KB )
      125. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/contract/LogHandlerInterface.php ( 0.86 KB )
      126. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/log/Channel.php ( 3.89 KB )
      127. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/event/LogRecord.php ( 1.02 KB )
      128. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-helper/src/Collection.php ( 16.47 KB )
      129. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/facade/View.php ( 1.70 KB )
      130. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/View.php ( 4.39 KB )
      131. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/Response.php ( 8.81 KB )
      132. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/response/View.php ( 3.29 KB )
      133. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/Cookie.php ( 6.06 KB )
      134. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-view/src/Think.php ( 8.38 KB )
      135. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/framework/src/think/contract/TemplateHandlerInterface.php ( 1.60 KB )
      136. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-template/src/Template.php ( 46.61 KB )
      137. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-template/src/template/driver/File.php ( 2.41 KB )
      138. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-template/src/template/contract/DriverInterface.php ( 0.86 KB )
      139. /yingpanguazai/ssd/ssd1/www/no.67808.cn/runtime/temp/6df755f970a38e704c5414acbc6e8bcd.php ( 12.06 KB )
      140. /yingpanguazai/ssd/ssd1/www/no.67808.cn/vendor/topthink/think-trace/src/Html.php ( 4.42 KB )
      1. CONNECT:[ UseTime:0.000970s ] mysql:host=127.0.0.1;port=3306;dbname=no_67808;charset=utf8mb4
      2. SHOW FULL COLUMNS FROM `fenlei` [ RunTime:0.001774s ]
      3. SELECT * FROM `fenlei` WHERE `fid` = 0 [ RunTime:0.000656s ]
      4. SELECT * FROM `fenlei` WHERE `fid` = 63 [ RunTime:0.000646s ]
      5. SHOW FULL COLUMNS FROM `set` [ RunTime:0.001466s ]
      6. SELECT * FROM `set` [ RunTime:0.000606s ]
      7. SHOW FULL COLUMNS FROM `article` [ RunTime:0.001566s ]
      8. SELECT * FROM `article` WHERE `id` = 471251 LIMIT 1 [ RunTime:0.000967s ]
      9. UPDATE `article` SET `lasttime` = 1772467401 WHERE `id` = 471251 [ RunTime:0.013309s ]
      10. SELECT * FROM `fenlei` WHERE `id` = 65 LIMIT 1 [ RunTime:0.002009s ]
      11. SELECT * FROM `article` WHERE `id` < 471251 ORDER BY `id` DESC LIMIT 1 [ RunTime:0.001170s ]
      12. SELECT * FROM `article` WHERE `id` > 471251 ORDER BY `id` ASC LIMIT 1 [ RunTime:0.002877s ]
      13. SELECT * FROM `article` WHERE `id` < 471251 ORDER BY `id` DESC LIMIT 10 [ RunTime:0.001932s ]
      14. SELECT * FROM `article` WHERE `id` < 471251 ORDER BY `id` DESC LIMIT 10,10 [ RunTime:0.010114s ]
      15. SELECT * FROM `article` WHERE `id` < 471251 ORDER BY `id` DESC LIMIT 20,10 [ RunTime:0.009570s ]
      0.207068s