/zh/authors/%E7%AE%97%E6%B3%95%E5%BC%80%E5%8F%912022/算法开发（2022）Wowchemy (https://wowchemy.com)zh-HansFri, 04 May 2018 08:58:25 +0000/media/icon_hua2ec155b4296a9c9791d015323e16eb5_11927_512x512_fill_lanczos_center_3.png/zh/authors/%E7%AE%97%E6%B3%95%E5%BC%80%E5%8F%912022//zh/project/cheetah_ros/Fri, 04 May 2018 08:58:25 +0000/zh/project/cheetah_ros/<h1 id="位姿估计与点云感知本科毕业论文吉大实验室">位姿估计与点云感知（本科毕业论文+吉大实验室）</h1> <h2 id="位姿估计">位姿估计</h2> <p>我的本科毕业论文聚焦于目标位姿估计。我以 RGB 图像和被检测目标的 3D 模型作为输入，将 2D 图像像素映射到模型表面的 3D 点云。在此基础上，我使用 PnP（Perspective-n-Point）和 RANSAC 算法回归目标位姿，实现目标位姿识别。同时，该方案还结合基于深度学习的 refinement 算法，在 PnP 得到初始位姿后进一步提升位姿精度。实验结果表明，与其他相关方法相比，建立 2D 平面与 3D 空间之间的映射关系，可以带来更高精度的位姿估计结果。</p> <p> <figure id="figure-整体算法框架设计"> <div class="d-flex justify-content-center"> <div class="w-100" ><img alt="整体算法框架设计" srcset=" /zh/project/cheetah_ros/paper_1_hu357f0a530b65630160c741efada1d310_324709_6e97fbed74fda6b046aed3fc8d78f383.webp 400w, /zh/project/cheetah_ros/paper_1_hu357f0a530b65630160c741efada1d310_324709_edcfb4f779e3b1122bc93017124ba17a.webp 760w, /zh/project/cheetah_ros/paper_1_hu357f0a530b65630160c741efada1d310_324709_1200x1200_fit_q75_h2_lanczos_3.webp 1200w" src="/zh/project/cheetah_ros/paper_1_hu357f0a530b65630160c741efada1d310_324709_6e97fbed74fda6b046aed3fc8d78f383.webp" width="760" height="234" loading="lazy" data-zoomable /></div> </div><figcaption> 整体算法框架设计 </figcaption></figure> </p> <p> <figure id="figure-使用随机位姿-3d-模型和-coco-背景生成虚拟数据集"> <div class="d-flex justify-content-center"> <div class="w-100" ><img alt="使用随机位姿 3D 模型和 COCO 背景生成虚拟数据集" srcset=" /zh/project/cheetah_ros/paper_2_hu4f1e6e92faede41be96d273333671e4c_574945_d051a94826179f1ff34aa711f79ab857.webp 400w, /zh/project/cheetah_ros/paper_2_hu4f1e6e92faede41be96d273333671e4c_574945_882a800f280033c4b16a37f023e0eb45.webp 760w, /zh/project/cheetah_ros/paper_2_hu4f1e6e92faede41be96d273333671e4c_574945_1200x1200_fit_q75_h2_lanczos_3.webp 1200w" src="/zh/project/cheetah_ros/paper_2_hu4f1e6e92faede41be96d273333671e4c_574945_d051a94826179f1ff34aa711f79ab857.webp" width="760" height="484" loading="lazy" data-zoomable /></div> </div><figcaption> 使用随机位姿 3D 模型和 COCO 背景生成虚拟数据集 </figcaption></figure> </p> <p> <figure id="figure-数据集生成结果"> <div class="d-flex justify-content-center"> <div class="w-100" ><img alt="数据集生成结果" srcset=" /zh/project/cheetah_ros/paper_3_huae6fedab5fb530819765175fcde398df_2210079_a1033ee5b4b730a4d37b93b05a4bcad2.webp 400w, /zh/project/cheetah_ros/paper_3_huae6fedab5fb530819765175fcde398df_2210079_164283e7a2ffc91decbf073e875eff75.webp 760w, /zh/project/cheetah_ros/paper_3_huae6fedab5fb530819765175fcde398df_2210079_1200x1200_fit_q75_h2_lanczos_3.webp 1200w" src="/zh/project/cheetah_ros/paper_3_huae6fedab5fb530819765175fcde398df_2210079_a1033ee5b4b730a4d37b93b05a4bcad2.webp" width="760" height="382" loading="lazy" data-zoomable /></div> </div><figcaption> 数据集生成结果 </figcaption></figure> </p> <p> <figure id="figure-uv-映射原理"> <div class="d-flex justify-content-center"> <div class="w-100" ><img alt="UV 映射原理" srcset=" /zh/project/cheetah_ros/paper_4_hu75509c5c488083500b00d3a3bc72eac9_58955_28378f2bef09ce2fdc46cac36572a637.webp 400w, /zh/project/cheetah_ros/paper_4_hu75509c5c488083500b00d3a3bc72eac9_58955_2886e33c068078623b9e3ac47b6159df.webp 760w, /zh/project/cheetah_ros/paper_4_hu75509c5c488083500b00d3a3bc72eac9_58955_1200x1200_fit_q75_h2_lanczos_3.webp 1200w" src="/zh/project/cheetah_ros/paper_4_hu75509c5c488083500b00d3a3bc72eac9_58955_28378f2bef09ce2fdc46cac36572a637.webp" width="592" height="540" loading="lazy" data-zoomable /></div> </div><figcaption> UV 映射原理 </figcaption></figure> </p> <p> <figure id="figure-uv-映射与物体表面点云之间的映射关系"> <div class="d-flex justify-content-center"> <div class="w-100" ><img alt="UV 映射与物体表面点云之间的映射关系" srcset=" /zh/project/cheetah_ros/paper_5_hu57e9dd53a2350e3005a00f000ebceabd_323873_1b32500e38121bf731ba490b4142bf60.webp 400w, /zh/project/cheetah_ros/paper_5_hu57e9dd53a2350e3005a00f000ebceabd_323873_b54dafd86542330efc48f8fc214384c5.webp 760w, /zh/project/cheetah_ros/paper_5_hu57e9dd53a2350e3005a00f000ebceabd_323873_1200x1200_fit_q75_h2_lanczos_3.webp 1200w" src="/zh/project/cheetah_ros/paper_5_hu57e9dd53a2350e3005a00f000ebceabd_323873_1b32500e38121bf731ba490b4142bf60.webp" width="760" height="388" loading="lazy" data-zoomable /></div> </div><figcaption> UV 映射与物体表面点云之间的映射关系 </figcaption></figure> </p> <p> <figure id="figure-uv-map-生成网络设计"> <div class="d-flex justify-content-center"> <div class="w-100" ><img alt="UV map 生成网络设计" srcset=" /zh/project/cheetah_ros/paper_7_hua72047cbbbc81b00d61c43aa15cd70ee_426901_0bc9dd5845b187065a19c19963ac8116.webp 400w, /zh/project/cheetah_ros/paper_7_hua72047cbbbc81b00d61c43aa15cd70ee_426901_02d29c5721c28a4cc0fd49c77b89e3dc.webp 760w, /zh/project/cheetah_ros/paper_7_hua72047cbbbc81b00d61c43aa15cd70ee_426901_1200x1200_fit_q75_h2_lanczos_3.webp 1200w" src="/zh/project/cheetah_ros/paper_7_hua72047cbbbc81b00d61c43aa15cd70ee_426901_0bc9dd5845b187065a19c19963ac8116.webp" width="760" height="504" loading="lazy" data-zoomable /></div> </div><figcaption> UV map 生成网络设计 </figcaption></figure> </p> <p> <figure id="figure-uv-map-生成结果"> <div class="d-flex justify-content-center"> <div class="w-100" ><img alt="UV map 生成结果" srcset=" /zh/project/cheetah_ros/paper_6_hu5b0b275728659814038e0df5c227d2fd_282644_0cf5a1abfdaabcf512452dcd9fe19c58.webp 400w, /zh/project/cheetah_ros/paper_6_hu5b0b275728659814038e0df5c227d2fd_282644_ed8d89b257d1657d7bad1eb952baae6d.webp 760w, /zh/project/cheetah_ros/paper_6_hu5b0b275728659814038e0df5c227d2fd_282644_1200x1200_fit_q75_h2_lanczos_3.webp 1200w" src="/zh/project/cheetah_ros/paper_6_hu5b0b275728659814038e0df5c227d2fd_282644_0cf5a1abfdaabcf512452dcd9fe19c58.webp" width="760" height="425" loading="lazy" data-zoomable /></div> </div><figcaption> UV map 生成结果 </figcaption></figure> </p> <p> <figure id="figure-uv-map-生成结果与标定图像对比"> <div class="d-flex justify-content-center"> <div class="w-100" ><img alt="UV map 生成结果与标定图像对比" srcset=" /zh/project/cheetah_ros/paper_8_huff366a0d115ad184c3c18d55457b2186_754847_e7804c717021cce50f268d145e91438e.webp 400w, /zh/project/cheetah_ros/paper_8_huff366a0d115ad184c3c18d55457b2186_754847_5e9dae638d94472434f13fb30ba2f70d.webp 760w, /zh/project/cheetah_ros/paper_8_huff366a0d115ad184c3c18d55457b2186_754847_1200x1200_fit_q75_h2_lanczos_3.webp 1200w" src="/zh/project/cheetah_ros/paper_8_huff366a0d115ad184c3c18d55457b2186_754847_e7804c717021cce50f268d145e91438e.webp" width="760" height="691" loading="lazy" data-zoomable /></div> </div><figcaption> UV map 生成结果与标定图像对比 </figcaption></figure> </p> <p> <figure id="figure-生成点云结果与标定点云对比"> <div class="d-flex justify-content-center"> <div class="w-100" ><img alt="生成点云结果与标定点云对比" srcset=" /zh/project/cheetah_ros/paper_9_hue1875f28882cc7ac3e1a8b68db573683_496787_9d80843363410945899129fc3b02bae4.webp 400w, /zh/project/cheetah_ros/paper_9_hue1875f28882cc7ac3e1a8b68db573683_496787_48996aadc729420eee895f08208e0a8b.webp 760w, /zh/project/cheetah_ros/paper_9_hue1875f28882cc7ac3e1a8b68db573683_496787_1200x1200_fit_q75_h2_lanczos_3.webp 1200w" src="/zh/project/cheetah_ros/paper_9_hue1875f28882cc7ac3e1a8b68db573683_496787_9d80843363410945899129fc3b02bae4.webp" width="760" height="613" loading="lazy" data-zoomable /></div> </div><figcaption> 生成点云结果与标定点云对比 </figcaption></figure> </p> <p> <figure id="figure-使用-ransacpnp-进行目标初始位姿回归的整体思路"> <div class="d-flex justify-content-center"> <div class="w-100" ><img alt="使用 RANSAC&#43;PnP 进行目标初始位姿回归的整体思路" srcset=" /zh/project/cheetah_ros/paper_10_hu2b730bb60ff33b5baff454b547df1461_747394_0c939ceb82a4b1c5d1b3c2c8f26cfa9f.webp 400w, /zh/project/cheetah_ros/paper_10_hu2b730bb60ff33b5baff454b547df1461_747394_485ca08bd3d9a2015b63a4c640c215d6.webp 760w, /zh/project/cheetah_ros/paper_10_hu2b730bb60ff33b5baff454b547df1461_747394_1200x1200_fit_q75_h2_lanczos_3.webp 1200w" src="/zh/project/cheetah_ros/paper_10_hu2b730bb60ff33b5baff454b547df1461_747394_0c939ceb82a4b1c5d1b3c2c8f26cfa9f.webp" width="760" height="365" loading="lazy" data-zoomable /></div> </div><figcaption> 使用 RANSAC+PnP 进行目标初始位姿回归的整体思路 </figcaption></figure> </p> <p> <figure id="figure-基于深度学习的位姿回归网络设计"> <div class="d-flex justify-content-center"> <div class="w-100" ><img alt="基于深度学习的位姿回归网络设计" srcset=" /zh/project/cheetah_ros/paper_11_hu7d06eb63b5ac49a418c2141920b71180_171665_64b9260e4c844b46947f5797ef45c41b.webp 400w, /zh/project/cheetah_ros/paper_11_hu7d06eb63b5ac49a418c2141920b71180_171665_5ac47c510aea57b0f16f05155377fa98.webp 760w, /zh/project/cheetah_ros/paper_11_hu7d06eb63b5ac49a418c2141920b71180_171665_1200x1200_fit_q75_h2_lanczos_3.webp 1200w" src="/zh/project/cheetah_ros/paper_11_hu7d06eb63b5ac49a418c2141920b71180_171665_64b9260e4c844b46947f5797ef45c41b.webp" width="760" height="500" loading="lazy" data-zoomable /></div> </div><figcaption> 基于深度学习的位姿回归网络设计 </figcaption></figure> </p> <p> <figure id="figure-位姿识别结果"> <div class="d-flex justify-content-center"> <div class="w-100" ><img alt="位姿识别结果" srcset=" /zh/project/cheetah_ros/paper_12_hu56de659dcb928fe4b729c6c003af8174_1943779_c7d5dd5bb64c6d50bc303b578ee22d2c.webp 400w, /zh/project/cheetah_ros/paper_12_hu56de659dcb928fe4b729c6c003af8174_1943779_7b034b66f4bddf4c8779b096fdfeb215.webp 760w, /zh/project/cheetah_ros/paper_12_hu56de659dcb928fe4b729c6c003af8174_1943779_1200x1200_fit_q75_h2_lanczos_3.webp 1200w" src="/zh/project/cheetah_ros/paper_12_hu56de659dcb928fe4b729c6c003af8174_1943779_c7d5dd5bb64c6d50bc303b578ee22d2c.webp" width="760" height="183" loading="lazy" data-zoomable /></div> </div><figcaption> 位姿识别结果 </figcaption></figure> </p> <h2 id="点云感知">点云感知</h2> <p>大四期间，我主要在吉林大学实验室开展点云感知相关工作。我完成了从 Livox 数据集到 KITTI 数据集的数据格式转换。</p> <p> <figure id="figure-数据格式转换"> <div class="d-flex justify-content-center"> <div class="w-100" ><img alt="数据格式转换" srcset=" /zh/project/cheetah_ros/livox_1_hu5c282198929f5668573023e9a8455cca_137500_0df2da9c219947824347822994e210a5.webp 400w, /zh/project/cheetah_ros/livox_1_hu5c282198929f5668573023e9a8455cca_137500_dd4873e8c548a3fc4af55591ce1f202d.webp 760w, /zh/project/cheetah_ros/livox_1_hu5c282198929f5668573023e9a8455cca_137500_1200x1200_fit_q75_h2_lanczos_3.webp 1200w" src="/zh/project/cheetah_ros/livox_1_hu5c282198929f5668573023e9a8455cca_137500_0df2da9c219947824347822994e210a5.webp" width="760" height="428" loading="lazy" data-zoomable /></div> </div><figcaption> 数据格式转换 </figcaption></figure> </p> <p>此外，我基于 Livox 数据集成功训练并实现了 PointPillars 模型的前向推理。</p> <p> <figure id="figure-前向推理与识别结果-1"> <div class="d-flex justify-content-center"> <div class="w-100" ><img alt="前向推理与识别结果 1" srcset=" /zh/project/cheetah_ros/livox_2_hu01ac4869239ccac0857a8e35c6fdefdc_584428_78523cee1153ffed511e0593eb0a0cdc.webp 400w, /zh/project/cheetah_ros/livox_2_hu01ac4869239ccac0857a8e35c6fdefdc_584428_954b7c1509976fa4677a754f4395157f.webp 760w, /zh/project/cheetah_ros/livox_2_hu01ac4869239ccac0857a8e35c6fdefdc_584428_1200x1200_fit_q75_h2_lanczos_3.webp 1200w" src="/zh/project/cheetah_ros/livox_2_hu01ac4869239ccac0857a8e35c6fdefdc_584428_78523cee1153ffed511e0593eb0a0cdc.webp" width="760" height="428" loading="lazy" data-zoomable /></div> </div><figcaption> 前向推理与识别结果 1 </figcaption></figure> <figure id="figure-前向推理与识别结果-2"> <div class="d-flex justify-content-center"> <div class="w-100" ><img alt="前向推理与识别结果 2" srcset=" /zh/project/cheetah_ros/livox_3_hud8509d13f1ff9b0c8920025fa164420e_565587_dba8618d5a8b86f803b7c390c4d66802.webp 400w, /zh/project/cheetah_ros/livox_3_hud8509d13f1ff9b0c8920025fa164420e_565587_d67cd91b87e67a0804cb2c1b2d6000c5.webp 760w, /zh/project/cheetah_ros/livox_3_hud8509d13f1ff9b0c8920025fa164420e_565587_1200x1200_fit_q75_h2_lanczos_3.webp 1200w" src="/zh/project/cheetah_ros/livox_3_hud8509d13f1ff9b0c8920025fa164420e_565587_dba8618d5a8b86f803b7c390c4d66802.webp" width="760" height="428" loading="lazy" data-zoomable /></div> </div><figcaption> 前向推理与识别结果 2 </figcaption></figure> </p> <p>通过这段经历，我系统了解了多种点云处理方法，也更清楚地认识到纯点云方法在实际目标识别中的局限性。这也激发了我继续探索点云与视觉感知融合的兴趣。</p> <p> <figure > <div class="d-flex justify-content-center"> <div class="w-100" ><img alt="" srcset=" /zh/project/cheetah_ros/livox_4_huceeb314967e847308dad09445d9ab574_470932_5672cb8bb2e583f9930142d8f20e0845.webp 400w, /zh/project/cheetah_ros/livox_4_huceeb314967e847308dad09445d9ab574_470932_3c0048defcbb6f1b5942ae1f063e0595.webp 760w, /zh/project/cheetah_ros/livox_4_huceeb314967e847308dad09445d9ab574_470932_1200x1200_fit_q75_h2_lanczos_3.webp 1200w" src="/zh/project/cheetah_ros/livox_4_huceeb314967e847308dad09445d9ab574_470932_5672cb8bb2e583f9930142d8f20e0845.webp" width="760" height="428" loading="lazy" data-zoomable /></div> </div></figure> <figure id="figure-纯点云方法在模型识别上仍有明显提升空间"> <div class="d-flex justify-content-center"> <div class="w-100" ><img alt="纯点云方法在模型识别上仍有明显提升空间" srcset=" /zh/project/cheetah_ros/livox_5_hu47e2f3b2021fb460cf3d5b61b902322f_374395_996d6d0514c464906c5cf0f668a461e4.webp 400w, /zh/project/cheetah_ros/livox_5_hu47e2f3b2021fb460cf3d5b61b902322f_374395_c7921ddb58e225be3bd8be4fdbf5b558.webp 760w, /zh/project/cheetah_ros/livox_5_hu47e2f3b2021fb460cf3d5b61b902322f_374395_1200x1200_fit_q75_h2_lanczos_3.webp 1200w" src="/zh/project/cheetah_ros/livox_5_hu47e2f3b2021fb460cf3d5b61b902322f_374395_996d6d0514c464906c5cf0f668a461e4.webp" width="760" height="428" loading="lazy" data-zoomable /></div> </div><figcaption> 纯点云方法在模型识别上仍有明显提升空间 </figcaption></figure> </p>