Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change Log generation tool. Automates change log generation. #2396

Merged
merged 6 commits into from
Oct 15, 2018

Conversation

SergioRAgostinho
Copy link
Member

@SergioRAgostinho SergioRAgostinho commented Aug 25, 2018

A tool written from Python3 to automate change log generation for each release. Here's the current reported usage with a Wiki article providing more details to follow.

$ python3 change_log.py -h
usage: change_log.py [-h] [--username USERNAME] [--cache [FILE] | --from-cache
                     [FILE]]
                     start [end]

Generate a change log between two revisions. Check
https://github.com/PointCloudLibrary/pcl/wiki/Preparing-Releases#creating-the-
change-log for some additional examples on how to use the tool.

positional arguments:
  start                The start (exclusive) revision/commit/tag to generate
                       the log.
  end                  The end (inclusive) revision/commit/tag to generate the
                       log. (Defaults to HEAD)

optional arguments:
  -h, --help           show this help message and exit
  --username USERNAME  GitHub Account user name. If specified it will perform
                       requests with the provided credentials. This is often
                       required in order to overcome GitHub API's request
                       limits.
  --cache [FILE]       Caches the PR info extracted from GitHub into a JSON
                       file. (Defaults to 'pr_info.json')
  --from-cache [FILE]  Uses a previously generated PR info JSON cache file to
                       generate the change log. (Defaults to 'pr_info.json')

Closes #2385

Here's the current generated output for my local clone.

= 1.8.1 (08.08.2017) =

New Features:

Newly added functionalities.

  • [octree] Implementation of the iterator 'OctreeLeafNodeBreadthIterator'. [#2204]
  • [octree] Implementation of the iterator 'OctreeFixedDepthIterator'. [#1983]
  • [ci] Enable Global Tests on Windows CI [#2137]
  • [features] Add GASD global point cloud descriptor [#1652]
  • [visualization] Add overload to PCLVisualizer::addText3D() that allows specifying text orientation [#2038]
  • [features] FLARELocalReferenceFrameEstimation class added [#1571]
  • [surface][tools] Add new mls projection method [#1960]

Deprecated:

Deprecated code scheduled to be removed after two minor releases.

  • [common][segmentation] Provide proper EuclideanClusterComparator method depreciation. New Pragma macro. New Deprecated type. [#2096]

Removed:

Removal of deprecated code.

  • [filters][io][surface][visualization] Removal of deprecated code in filters, io, surface and visualization modules [#2077]
  • [common] Remove deprecated ros headers [#2075]
  • [registration] Remove registration module deprecated methods [#2076]
  • [sample_consensus] Remove deprecated functions and variables from SAC module [#2071]
  • [common] Removal of PCA deprecated constructor [#2070]
  • [segmentation] Removed normal related accessors and types from EuclideanClusterComparator [#1542]

Behavioral changes:

Changes in the expected default behavior.

  • [io] Fix PLYReader is_dense behavior [#2133]

API changes:

Changes to the API which didn't went through the proper deprecation and removal cycle.

  • [simulation] Add const qualifiers to RangeLikelihood getters. [#2411]
  • [octree] Implementation of the iterator 'OctreeLeafNodeBreadthIterator'. [#2204]
  • [surface] Missing pcl::MovingLeastSquaresOMP declaration without /openmp [#2324]
  • [sample_consensus] Const-qualify most of the methods in SAC model classes [#2270]
  • [simulation] Use GLuint rather than size_t to represent OpenGL indices. [#2238]
  • [visualization] Fix access specifier in PointCloudColorHandlerRGBAField [#2226]
  • [docs] Misc. typos (cont.) [#2215]
  • [filters] Add const qualifiers to getters in filter_indices.h [#2193]
  • [octree] Remove parametrization of end iterators [#2168]
  • [octree] OctreeIterators special member revision [#2108]
  • [io] Add support pcl::PointXYZRGBA to pcl::VLPGrabber [#2102]
  • [filters][io][surface][visualization] Removal of deprecated code in filters, io, surface and visualization modules [#2077]
  • [common] Remove deprecated ros headers [#2075]
  • [registration] Remove registration module deprecated methods [#2076]
  • [sample_consensus] Remove deprecated functions and variables from SAC module [#2071]
  • [common] Removal of PCA deprecated constructor [#2070]
  • [segmentation] Removed normal related accessors and types from EuclideanClusterComparator [#1542]
  • [surface][tools] Add new mls projection method [#1960]
  • [surface] Add ability to cache mls results [#1952]

ABI changes:

Changes that cause ABI incompatibility but are still API compatible.

  • [common] Modified GlasbeyLUT indexing type to size_t [#2297]
  • [octree] Implementation of the iterator 'OctreeFixedDepthIterator'. [#1983]
  • [common][segmentation] Provide proper EuclideanClusterComparator method depreciation. New Pragma macro. New Deprecated type. [#2096]
  • [gpu] Allow specifying decimation step in convertToTsdfCloud [#2099]
  • [apps] More warning suppression in pcl apps [#2080]
  • [io] Removed unused member from ply_parser [#2066]
  • [filters] Fixes remove_indices in UniformSampling [#1902]
  • [visualization] Add accessor for current rendering framerate in PCLVisualizer [#1974]
  • [simulation] Redo: Simulation: enable returning of organized point clouds [#1687]
  • [registration] Added option to specify translation and rotation convergence deltas in ICP and NDT algorithms. [#1724]

Modules:

Uncategorized:

  • Never close stale issues/prs [#2400]
  • Fix typos in the whole codebase [#2217]
  • Fixed typo and rearragend items in the issue template [#2197]
  • Change Stale daysTillClose to 100 years [#2166]
  • set stale daysUntilClose to a really big number [#2162]
  • Stale set up [#2101]

CMake:

  • Move SSE compiler options to PCL_COMPILE_OPTIONS. Expose PCL as a CMake imported target. [#2100]
  • Add Visual Studio compiler option /FS for Ninja build [#2414]
  • Use rpath in the target's install name [#2241]
  • Improve QHull finder script [#2344]
  • Fix link order issue with boost [#2236]
  • Mark found PCL component libraries and include dirs as advanced [#2235]
  • Prevent search for disabled optional dependencies in targets. [#2229]
  • Fix installation rules for ml module [#2192]
  • Fix conditional branch of Visual C++ 2017 [#2121]
  • Add *_USE_STATIC options to PCLConfig [#2086]
  • Add search path suffixes for Vcpkg [#2085]
  • Update finder scripts for Ensenso, OpenNI, and OpenNI2 [#2061]
  • Fix PACKAGE to include cmake/Modules directory [#2053]
  • Unifies Find scripts in PCLConfig [#1421]
  • CUDA 9 Arch Flags [#2047]
  • Suppress log when PCL_FIND_QUIETLY is turned on. [#2032]
  • fix /MP option not generated for Visual Studio. [#2031]
  • Generate pkgconfig for 2d module [#1979]
  • Update Find Boost [#1972]
  • Added CUDA compute capability 5.3 [#1929]
  • Fix issue with finding pcl deployed out of path [#1923]
  • Add new gtest path [#1920]

libpcl_2d:

  • Avoid huge index jumps in RandomSample. Remove io dependency from 2d. [#2141]
  • Fix header names [#2079]
  • Generate pkgconfig for 2d module [#1979]

libpcl_common:

  • Remove malloc header to restore builds on BSDs [#2374]
  • Add support for multiple extensions in parse_file_extension_argument (). [#2347]
  • Improve speed of transformPointCloud/WithNormals() functions [#2247]
  • Add RGB constructor that takes R, G, and B components [#2329]
  • Improved docstrings and error messages [#2300]
  • [abi] Modified GlasbeyLUT indexing type to size_t [#2297]
  • Add GASDSignatures to PCL_POINT_TYPES and PCL_FEATURE_POINTTYPES macros. [#2295]
  • [PARSE] Constness of the API [#2224]
  • Fix two "unreachable code" warnings in pca.hpp [#2219]
  • Fix covariance calculation in PCA [#2130]
  • [abi][deprecation] Provide proper EuclideanClusterComparator method depreciation. New Pragma macro. New Deprecated type. [#2096]
  • [api][removal] Remove deprecated ros headers [#2075]
  • Suppress (maybe) uninitialized warning [#2073]
  • [api][removal] Removal of PCA deprecated constructor [#2070]
  • [gcc] fixes -Wimplicit-fallthrough: common/io.h [#2041]
  • Include pcl/point_cloud.h and pcl/point_types.h headers. [#1962]
  • Add test for macro _USE_MATH_DEFINES. [#1956]
  • instantiate: remove duplicate macro definition. Fixes duplicate macro definition in instantiate.hpp header #1924. [#1925]

libpcl_cuda:

  • Fix compilation issues with CUDA 9.1 [#2212]
  • Fix some CUDA 9 related errors [#2181]
  • Added CUDA compute capability 5.3 [#1929]

libpcl_features:

  • Update the OpenMP implementations of normal and FPFH estimation [#2278]
  • Make MomentOfInertia instantiations consistent with the rest of the library [#2266]
  • Docstring corrections [#2143]
  • Improve Doxygen comments for HistogramInterpolationMethod [#2111]
  • [new-feature] Add GASD global point cloud descriptor [#1652]
  • Suppress (maybe) uninitialized warning [#2073]
  • [new-feature] FLARELocalReferenceFrameEstimation class added [#1571]
  • fix missing include file: from_meshes.h is using pcl::Vertices in it [#2009]
  • Typo [#1968]

libpcl_filters:

  • Add PointNormal to ExtractIndices Instantiate Types [#2389]
  • Improved docstrings and error messages [#2300]
  • Public access to VoxelGrid boost pointer. [#2205]
  • [api] Add const qualifiers to getters in filter_indices.h [#2193]
  • Avoid huge index jumps in RandomSample. Remove io dependency from 2d. [#2141]
  • [api][removal] Removal of deprecated code in filters, io, surface and visualization modules [#2077]
  • Suppress unused parameter warning [#2074]
  • Suppress sign compare warnings [#2068]
  • Transformation Fix for BoxClipper3D [#1961]
  • [abi] Fixes remove_indices in UniformSampling [#1902]
  • Inherit StatisticalOutlierRemoval from FilterIndices [#1663]

libpcl_gpu:

libpcl_io:

  • pcl::PCDReader::readHeader() change nr_points type to size_t to avoid possible int32 overflow [#2408]
  • Fix raw_fallocate for Android and deal with unsupported filesystems. [#2363]
  • Add low_level_io.h to the header list of the io module [#2356]
  • Created header for low level I/O helpers. Fix for ::posix_fallocate on Mac OSX [#2354]
  • Added warnings when the input data is too large for compressed pcds [#2323]
  • Allocate disk space with posix_fallocate before mmapping. [#2325]
  • Fix cmake warning: Logical block closes with mis-matching arguments [#2320]
  • Added PCL_IO_ENABLE_MAND_LOCKING cmake flag. [#2315]
  • Added missing 8 bytes to compressed binary pcd length. [#2281]
  • Remove useless size check in PLYReader::endHeaderCallback() [#2246]
  • [behavior] Fix PLYReader is_dense behavior [#2133]
  • EnsensoGrabber uint is undefined in Visual studio. [#2223]
  • Add protection from invalid WIDTH values in PCD reader [#2195]
  • PLYReader Cast cloud point step as 64-bit integer [#2161]
  • OpenNI2Device Add device sensor check for IR and depth modesetting [#2150]
  • Adds a check for when CreateFileMappingA fails [#2146]
  • PCDWriterchanged toff to size_t in writeBinaryCompressed [#2144]
  • Prevent POINTS field parsing before point_step is specified [#2131]
  • Check COUNT value specified in PCD files [#2126]
  • Prevent mmapping more than the original PCD file size [#2125]
  • [api] Add support pcl::PointXYZRGBA to pcl::VLPGrabber [#2102]
  • [api][removal] Removal of deprecated code in filters, io, surface and visualization modules [#2077]
  • Suppress strict alias warning [#2072]
  • Suppress unused parameter warnings [#2067]
  • [abi] Removed unused member from ply_parser [#2066]
  • Suppress control reaches end of non-void function in io.h [#2057]
  • Modify STRICT_ALIGN because macro expansion w/defined is undefined [#2043]
  • Add necessary boost headers to pcl/io to build in CUDA mode [#2025]
  • Fix MSVC compile issue related with ssize_t [#2027]
  • Adds in-memory PCD serialization/deserialization; de-duplicates PCDReader::readHeader(). (take Fix compilation #2) [#1986]

libpcl_kdtree:

  • Consistent ptr typedefs for kd tree flann [#1607]

libpcl_keypoints:

  • Add TrajkovicKeypoint2D/3D to CMake build [#2179]

libpcl_ml:

  • Fix installation rules for ml module [#2192]

libpcl_octree:

  • octree_key.h suppress GCC 8 ignored-qualifier warning [#2405]
  • [api][new-feature] Implementation of the iterator 'OctreeLeafNodeBreadthIterator'. [#2204]
  • [abi][new-feature] Implementation of the iterator 'OctreeFixedDepthIterator'. [#1983]
  • Octree Iterator begin/end method and added tests [#2174]
  • [api] Remove parametrization of end iterators [#2168]
  • Fix docstrings in octree test [#2173]
  • [api] OctreeIterators special member revision [#2108]
  • Remove unused variable from octree_viewer [#2069]
  • Silence compile warning by removing superfluous call to std::max() [#2014]
  • [OCTREE] Add bounding box checks in isVoxelOccupiedAtPoint() and deleteVoxelAtPoint() [#1976]

libpcl_outofcore:

  • Explictly use mt19937 random generator for boost 1.67. [#2338]
  • Fixed queryBBIncludes_subsample [#1988]

libpcl_people:

libpcl_recognition:

  • Setting the resolution of the occupancy grid [#2273]
  • Inline helper function gcCorrespSorter() [#2248]
  • Misleading indentation [#2034]

libpcl_registration:

  • [api][removal] Remove registration module deprecated methods [#2076]
  • Suppress (maybe) uninitialized warning [#2073]
  • Remove unreachable code in DefaultConvergenceCriteria [#1967]
  • [abi] Added option to specify translation and rotation convergence deltas in ICP and NDT algorithms. [#1724]

libpcl_sample_consensus:

  • [api] Const-qualify most of the methods in SAC model classes [#2270]
  • [api][removal] Remove deprecated functions and variables from SAC module [#2071]

libpcl_search:

  • Correct testPoint for organized nearest neighbor search [#2198]

libpcl_segmentation:

  • Avoid infinite recursion in getPointCloudDifference [#2402]
  • Correct setting of is_dense flag in SegmentDifferences. Deprecate unused parameter in method. [#2380]
  • Dereference shared_ptr, fix for GCC8 [#2299]
  • [abi][deprecation] Provide proper EuclideanClusterComparator method depreciation. New Pragma macro. New Deprecated type. [#2096]
  • [api][removal] Removed normal related accessors and types from EuclideanClusterComparator [#1542]

libpcl_simulation:

  • [api] Add const qualifiers to RangeLikelihood getters. [#2411]
  • [api] Use GLuint rather than size_t to represent OpenGL indices. [#2238]
  • Support both RGB and RGBA colors in mesh loading [#2036]
  • [abi] Redo: Simulation: enable returning of organized point clouds [#1687]
  • Simulation: more access to camera parameters [#1650]

libpcl_surface:

  • Make pcl::MovingLeastSquares thread-safe [#2418]
  • [api] Missing pcl::MovingLeastSquaresOMP declaration without /openmp [#2324]
  • Improved docstrings and error messages [#2300]
  • opennurbs: fix ON_Curve::EvaluatePoint calculation [#2185]
  • [api][removal] Removal of deprecated code in filters, io, surface and visualization modules [#2077]
  • Suppress (maybe) uninitialized warning [#2073]
  • Suppress sign compare warnings [#2068]
  • Fix incorrect Ptr/ConstPtr typedefs in MovingLeastSquaresOMP [#2055]
  • Replace float indices with Eigen Index [#2017]
  • [api][new-feature] Add new mls projection method [#1960]
  • Avoid phantom surfces in marching cubes hoppe [#1874]
  • [api] Add ability to cache mls results [#1952]

PCL Apps:

  • add windows.h for includes GL/gl.h; handle cancel for denoiseWidget. [#2267]
  • Add missing dependecy to apps [#2251]
  • Suppress the final set of warnings in pcl apps [#2082]
  • [abi] More warning suppression in pcl apps [#2080]

PCL Docs:

  • Switch to using client-based search in Doxygen [#2391]
  • [api] Misc. typos (cont.) [#2215]
  • doc: misc. typos [#2213]
  • Add url to API/ABI compatibity report [#2116]
  • Improve Doxygen comments for HistogramInterpolationMethod [#2111]
  • Update organized.h [#1965]
  • Typo [#1968]
  • Fixed spelling and grammar errors [#1959]
  • Fixed error in documentation. [#1957]

PCL Tutorials:

  • Add settings for hypothesis verification [#2274]
  • Fix ICP tutorial [#2244]
  • Fix error in example code for estimate set of surface for a subset of points in the input dataset [#2203]
  • Update message about legacy point cloud types in tutorial [#2175]
  • Add descriptor unpacking to GASD tutorial [#2167]
  • Fix convert to Eigen::Map<const Eigen::Vector3f> from Normal of pcl::PointXYZINormal [#2128]
  • Fix the tutorial qt_visualizer compilation issue: qt4 -> qt5. [#2051]
  • Fix several documentation typos [#2020]
  • Replace literal include of wrong CMakeLists file with correct script [#1971]
  • Update Ensenso tutorial for Ensenso X devices [#1933]

PCL Examples:

  • Suppress strict alias warning [#2072]
  • Suppress (maybe) uninitialized warning [#2073]
  • Fix CPC/LCCP segmentation examples for VTK 7.1 [#2063]

PCL Tests:

  • Cleanup and improve unit test coverage for transformPointCloud functions [#2245]
  • Fixes and new assertion macro in "pcl_tests.h" [#2237]
  • Add new gtest path [#1920]

PCL Tools:

  • Allow the pcl_uniform_sampling tool to deal with several formats (PCD, PLY and VTK) as input or output point cloud [#2348]
  • mesh_sampling tool: Add support for colors [#2257]
  • Error message on non-recognized feature names [#2178]
  • Suppress sign compare warnings [#2068]
  • [OCTREE] Compute accurately the centroids of octree in 'pcl_octree_viewer' [#1981]
  • [api][new-feature] Add new mls projection method [#1960]
  • [OCTREE] Fix pcl_octree_viewer [#1973]
  • [OCTREE] Remove a useless field in octree_viewer. [#1980]

CI:

  • Split (yet again) Travis test job into two and tweak timings in building apps [#2182]
  • [new-feature] Enable Global Tests on Windows CI [#2137]
  • Travis merge test jobs. Expose VTK include directory for the visualization module. [#2163]
  • Remove unnecessary PPAs and packages from Travis [#2153]
  • AppVeyor - Change to simple style of specify triplet [#2135]
  • Initial Appveyor CI integration [#2083]
  • Change Travis to use pip3 for installing sphinx [#2124]
  • [TRAVIS] Enable the build of apps. [#2012]
  • [TRAVIS] Enable the build of tools. [#2007]
  • Disable tools build in CI. [#2003]

@taketwo
Copy link
Member

taketwo commented Aug 25, 2018

Looks great! Couple of minor comments:

  • This script has minor incompatibility with Python 3.5 because you use Path objects as arguments to subprocess.run. It's easy to fix by wrapping them: str(FOLDER).
  • Maybe put this script into .github folder? Our top-level folders are (ideally) reserved for library modules.
  • Pass the script through a formatter? I'd go with Black.

@SergioRAgostinho
Copy link
Member Author

Maybe put this script into .github folder? Our top-level folders are (ideally) reserved for library modules.

I'm uncertain if this is the best option. These are maintenance tools which don't really have much to do with GitHub. Yet one could argue "there's a reason why we're generating markdown changelogs with hashtags to PRs". I also know that one of the next steps will be to write a bash script to produce the Compatibility report between versions. This last one has nothing to do with GitHub.

How about creating a .releases or .maintenance folder, using the folder visibility to convey the idea that is not a module?

@SergioRAgostinho
Copy link
Member Author

SergioRAgostinho commented Aug 25, 2018

Just finished the Wiki article. Have a look at it.

https://github.com/PointCloudLibrary/pcl/wiki/Preparing-Releases#creating-the-change-log

Next step: labelling PRs.

Edit: Just realized it's been almost 3-5 months since I looked at c++.

Edit2: I just realized we now can print cool stats like the top 5 modules which suffered changes or the top 5 PR authors which submitted fixes.

@taketwo
Copy link
Member

taketwo commented Aug 26, 2018

On one hand .releases or .maintenance makes sense, but on the other hand it's somewhat unconventional (never saw something like this). Since you already moved it to .github I'd leave it there for the time being.

👍 for the comprehensive Wiki page!

Do you think CI-related things need to go to the changelog?

@SergioRAgostinho
Copy link
Member Author

Do you think CI-related things need to go to the changelog?

Answering your question: no, but it's already there. I think I enjoy more having it than not having it.

@taketwo
Copy link
Member

taketwo commented Aug 27, 2018

OK. Perhaps you may want to fix or change something in this script during the release process, so probably want to leave this open until after the release, right?

@SergioRAgostinho SergioRAgostinho added this to the pcl-1.9.0 milestone Aug 27, 2018
@SergioRAgostinho
Copy link
Member Author

SergioRAgostinho commented Aug 27, 2018

Exactly. I was expecting to merge right before the last change log step in release task list.

Note to self: Don't forget to include #1652 in the change log.

@taketwo
Copy link
Member

taketwo commented Aug 31, 2018

I realized I'm not entirely satisfied with the summary. I believe it should clearly answer the following questions:

  1. What's new? New features that I will get and can try out.
  2. Any surprises?
    1. Behavior changes that may affect my existing code.
    2. API changes that did not go through proper deprecation cycle.
  3. What's newly deprecated? What should I modify to ensure compatibility with future releases.
  4. What deprecated stuff was removed? What will no longer work.

1 and 2i are covered, however there is no distinction between the rest. Also, for example, #2380 is completely missing from the list, I suppose because we don't have (yet) a good label for it since it's not an API break.

By the way, some food for thought.

@SergioRAgostinho
Copy link
Member Author

SergioRAgostinho commented Sep 2, 2018

I think I would reorder your points to 1, 3, 4, 2. Rationale: removal of deprecated things always causes api changes and that way in point 2 we only mention things which didn't appear in 4. Meaning that in point 2 only things without proper deprecation cycle can appear. It's like we're guiding the process from good news to bad news.

To account for these new categories I propose the creation of two new labels: changes: deprecation and changes: removal. changes: removal will superseed changes: breaks API since it immediately implies an API breakage.

Let's agree on an output text dump which matches the new bullet points.

Change Log

What's new?

Things under the new featurelabel

What's newly deprecated?

Things under changes: deprecated label.

What deprecated stuff was removed?

Things under changes: removal label.

Any surprises?

Changes in Behavior

Things under changes: behavior.

API Breakage

Things under changes: breaks API.

[ABI Breakage]

[3. Consider adding things changes: breaks ABI. It should be ok but who knows.]

Modules

[...]


There's two extra points it might be worth a retweak.

  1. I noticed a PR which didn't appear in the change log because the merge message was modified. I'm going to rewrite the script logic to look into all merge commits to master. The ones which follow the usual merge master I'll extract the PR number the same way. The ones which don't I'll try to parse the message for matching for a single #d string. If there's no single occurrence of #d string in the message or there's more than one, I think the only option is to prompt the user for help and have him manually inserting the PR number. Hopefully this is limited to one or two cases.
  2. Based on your linked article, I also thought about a grouping mechanism for PRs. For instance: image we have 2 or more PRs related to a new registration method which is first created in PR #X and then it gets some minor corrections in #Y. We name #X with the appropriate text we want to render on the change log and in the title of PR #Y we add a small tag like [parent #X]. During change log generation, both PRs will associated and PR #X will inherit the labels from its children. I must say that I'm not very eager to revisit all PRs again an decided which ones should be grouped or not. I think would prefer to have this in mind for the next release and start tagging/naming PRs with this strategy.

@SergioRAgostinho
Copy link
Member Author

Solution to 1. was a little more simple than initially conjectured. Updated the change log. There's more stuff to categorize now.

@taketwo
Copy link
Member

taketwo commented Sep 2, 2018

👍 for the proposed ordering. I'd probably go with more formal names for the sections, e.g.

  • what's new -> new features
  • what's newly deprecated -> deprecated
  • what deprecated stuff was removed -> removed
  • any surprises? -> behavioral/API/ABI changes

Also, we can consider adding one-two sentences in the beginning of each section to explain it's meaning.

Extra point 2: I think for this release we can do a bit of grouping and clean-up of the generated list manually just before releasing. In future a convention like you described could be helpful. Maybe we wait until situation like this occurs and decide on the approach?

Finally, I added labels to the most of uncategorized PRs. (But you've noticed this already, I suppose).

@SergioRAgostinho
Copy link
Member Author

Have a look at the section reordering and the descriptions. We might need recheck the API/removal changes. I'm looking forward to close the change log for this release.

@taketwo
Copy link
Member

taketwo commented Sep 9, 2018

Thanks, LGTM

@SergioRAgostinho
Copy link
Member Author

SergioRAgostinho commented Sep 25, 2018

Latest Log (click to uncollapse)

= master (25.09.2018) =

New Features:

Newly added functionalities.

  • [common][visualization] Add Viridis color LUT [#2420]
  • [octree] Implementation of the iterator 'OctreeLeafNodeBreadthIterator'. [#2204]
  • [octree] Implementation of the iterator 'OctreeFixedDepthIterator'. [#1983]
  • [ci] Enable Global Tests on Windows CI [#2137]
  • [features] Add GASD global point cloud descriptor [#1652]
  • [visualization] Add overload to PCLVisualizer::addText3D() that allows specifying text orientation [#2038]
  • [features] FLARELocalReferenceFrameEstimation class added [#1571]
  • [surface][tools] Add new mls projection method [#1960]

Deprecated:

Deprecated code scheduled to be removed after two minor releases.

  • [common][segmentation] Provide proper EuclideanClusterComparator method depreciation. New Pragma macro. New Deprecated type. [#2096]

Removed:

Removal of deprecated code.

  • [filters][io][surface][visualization] Removal of deprecated code in filters, io, surface and visualization modules [#2077]
  • [common] Remove deprecated ros headers [#2075]
  • [registration] Remove registration module deprecated methods [#2076]
  • [sample_consensus] Remove deprecated functions and variables from SAC module [#2071]
  • [common] Removal of PCA deprecated constructor [#2070]

Behavioral changes:

Changes in the expected default behavior.

  • [io] Fix PLYReader is_dense behavior [#2133]

API changes:

Changes to the API which didn't went through the proper deprecation and removal cycle.

  • [simulation] Add const qualifiers to RangeLikelihood getters. [#2411]
  • [octree] Implementation of the iterator 'OctreeLeafNodeBreadthIterator'. [#2204]
  • [surface] Missing pcl::MovingLeastSquaresOMP declaration without /openmp [#2324]
  • [sample_consensus] Const-qualify most of the methods in SAC model classes [#2270]
  • [simulation] Use GLuint rather than size_t to represent OpenGL indices. [#2238]
  • [visualization] Fix access specifier in PointCloudColorHandlerRGBAField [#2226]
  • [docs] Misc. typos (cont.) [#2215]
  • [filters] Add const qualifiers to getters in filter_indices.h [#2193]
  • [octree] Remove parametrization of end iterators [#2168]
  • [octree] OctreeIterators special member revision [#2108]
  • [io] Add support pcl::PointXYZRGBA to pcl::VLPGrabber [#2102]
  • [surface][tools] Add new mls projection method [#1960]
  • [surface] Add ability to cache mls results [#1952]

ABI changes:

Changes that cause ABI incompatibility but are still API compatible.

  • [common] Modified GlasbeyLUT indexing type to size_t [#2297]
  • [octree] Implementation of the iterator 'OctreeFixedDepthIterator'. [#1983]
  • [common][segmentation] Provide proper EuclideanClusterComparator method depreciation. New Pragma macro. New Deprecated type. [#2096]
  • [gpu] Allow specifying decimation step in convertToTsdfCloud [#2099]
  • [apps] More warning suppression in pcl apps [#2080]
  • [io] Removed unused member from ply_parser [#2066]
  • [filters] Fixes remove_indices in UniformSampling [#1902]
  • [visualization] Add accessor for current rendering framerate in PCLVisualizer [#1974]
  • [simulation] Redo: Simulation: enable returning of organized point clouds [#1687]
  • [registration] Added option to specify translation and rotation convergence deltas in ICP and NDT algorithms. [#1724]

Modules:

Uncategorized:

  • Fix link to Institut Maupertuis's ensenso_extrinsic_calibration repo [#2447]
  • Never close stale issues/prs [#2400]
  • Fix typos in the whole codebase [#2217]
  • Fixed typo and rearragend items in the issue template [#2197]
  • Change Stale daysTillClose to 100 years [#2166]
  • set stale daysUntilClose to a really big number [#2162]
  • Stale set up [#2101]

CMake:

  • CMake Set temporarily the policy CMP0074 to OLD [#2454]
  • prevent GCC flags propagating to NVCC [#2430]
  • Mark visualization as an optional dependency of tools [#2439]
  • Do not mark imported libraries as GLOBAL in PCLConfig [#2435]
  • Intel fixes [#2432]
  • Export -march=native for Clang and prevent it from being included during cross compilation. [#2416]
  • Do not search for PCL components that have been found already [#2428]
  • Move SSE compiler options to PCL_COMPILE_OPTIONS. Expose PCL as a CMake imported target. [#2100]
  • Add Visual Studio compiler option /FS for Ninja build [#2414]
  • Use rpath in the target's install name [#2241]
  • Improve QHull finder script [#2344]
  • Fix link order issue with boost [#2236]
  • Mark found PCL component libraries and include dirs as advanced [#2235]
  • Prevent search for disabled optional dependencies in targets. [#2229]
  • Fix installation rules for ml module [#2192]
  • Fix conditional branch of Visual C++ 2017 [#2121]
  • Add *_USE_STATIC options to PCLConfig [#2086]
  • Add search path suffixes for Vcpkg [#2085]
  • Update finder scripts for Ensenso, OpenNI, and OpenNI2 [#2061]
  • Fix PACKAGE to include cmake/Modules directory [#2053]
  • Unifies Find scripts in PCLConfig [#1421]
  • CUDA 9 Arch Flags [#2047]
  • Suppress log when PCL_FIND_QUIETLY is turned on. [#2032]
  • fix /MP option not generated for Visual Studio. [#2031]
  • Generate pkgconfig for 2d module [#1979]
  • Update Find Boost [#1972]
  • Added CUDA compute capability 5.3 [#1929]
  • Fix issue with finding pcl deployed out of path [#1923]
  • Add new gtest path [#1920]

libpcl_2d:

  • Avoid huge index jumps in RandomSample. Remove io dependency from 2d. [#2141]
  • Fix header names [#2079]
  • Generate pkgconfig for 2d module [#1979]

libpcl_common:

  • Add some missing eigen alignment operators [#2433]
  • Intel fixes [#2432]
  • [new-feature] Add Viridis color LUT [#2420]
  • Remove malloc header to restore builds on BSDs [#2374]
  • Add support for multiple extensions in parse_file_extension_argument (). [#2347]
  • Improve speed of transformPointCloud/WithNormals() functions [#2247]
  • Add RGB constructor that takes R, G, and B components [#2329]
  • Improved docstrings and error messages [#2300]
  • [abi] Modified GlasbeyLUT indexing type to size_t [#2297]
  • Add GASDSignatures to PCL_POINT_TYPES and PCL_FEATURE_POINTTYPES macros. [#2295]
  • [PARSE] Constness of the API [#2224]
  • Fix two "unreachable code" warnings in pca.hpp [#2219]
  • Fix covariance calculation in PCA [#2130]
  • [abi][deprecation] Provide proper EuclideanClusterComparator method depreciation. New Pragma macro. New Deprecated type. [#2096]
  • [removal] Remove deprecated ros headers [#2075]
  • Suppress (maybe) uninitialized warning [#2073]
  • [removal] Removal of PCA deprecated constructor [#2070]
  • [gcc] fixes -Wimplicit-fallthrough: common/io.h [#2041]
  • Include pcl/point_cloud.h and pcl/point_types.h headers. [#1962]
  • Add test for macro _USE_MATH_DEFINES. [#1956]
  • instantiate: remove duplicate macro definition. Fixes duplicate macro definition in instantiate.hpp header #1924. [#1925]

libpcl_cuda:

  • Fix compilation issues with CUDA 9.1 [#2212]
  • Fix some CUDA 9 related errors [#2181]
  • Added CUDA compute capability 5.3 [#1929]

libpcl_features:

  • Update the OpenMP implementations of normal and FPFH estimation [#2278]
  • Make MomentOfInertia instantiations consistent with the rest of the library [#2266]
  • Docstring corrections [#2143]
  • Improve Doxygen comments for HistogramInterpolationMethod [#2111]
  • [new-feature] Add GASD global point cloud descriptor [#1652]
  • Suppress (maybe) uninitialized warning [#2073]
  • [new-feature] FLARELocalReferenceFrameEstimation class added [#1571]
  • fix missing include file: from_meshes.h is using pcl::Vertices in it [#2009]
  • Typo [#1968]

libpcl_filters:

  • Add some missing eigen alignment operators [#2433]
  • Add PointNormal to ExtractIndices Instantiate Types [#2389]
  • Improved docstrings and error messages [#2300]
  • Public access to VoxelGrid boost pointer. [#2205]
  • [api] Add const qualifiers to getters in filter_indices.h [#2193]
  • Avoid huge index jumps in RandomSample. Remove io dependency from 2d. [#2141]
  • [removal] Removal of deprecated code in filters, io, surface and visualization modules [#2077]
  • Suppress unused parameter warning [#2074]
  • Suppress sign compare warnings [#2068]
  • Transformation Fix for BoxClipper3D [#1961]
  • [abi] Fixes remove_indices in UniformSampling [#1902]
  • Inherit StatisticalOutlierRemoval from FilterIndices [#1663]

libpcl_gpu:

libpcl_io:

  • Improved obj file parsing efficiency. Make parsing robust against situations where there are more normals than points. Added unit tests. [#2450]
  • pcl::PCDReader::readHeader() change nr_points type to size_t to avoid possible int32 overflow [#2408]
  • Fix raw_fallocate for Android and deal with unsupported filesystems. [#2363]
  • Add low_level_io.h to the header list of the io module [#2356]
  • Created header for low level I/O helpers. Fix for ::posix_fallocate on Mac OSX [#2354]
  • Added warnings when the input data is too large for compressed pcds [#2323]
  • Allocate disk space with posix_fallocate before mmapping. [#2325]
  • Fix cmake warning: Logical block closes with mis-matching arguments [#2320]
  • Added PCL_IO_ENABLE_MAND_LOCKING cmake flag. [#2315]
  • Added missing 8 bytes to compressed binary pcd length. [#2281]
  • Remove useless size check in PLYReader::endHeaderCallback() [#2246]
  • [behavior] Fix PLYReader is_dense behavior [#2133]
  • EnsensoGrabber uint is undefined in Visual studio. [#2223]
  • Add protection from invalid WIDTH values in PCD reader [#2195]
  • PLYReader Cast cloud point step as 64-bit integer [#2161]
  • OpenNI2Device Add device sensor check for IR and depth modesetting [#2150]
  • Adds a check for when CreateFileMappingA fails [#2146]
  • PCDWriterchanged toff to size_t in writeBinaryCompressed [#2144]
  • Prevent POINTS field parsing before point_step is specified [#2131]
  • Check COUNT value specified in PCD files [#2126]
  • Prevent mmapping more than the original PCD file size [#2125]
  • [api] Add support pcl::PointXYZRGBA to pcl::VLPGrabber [#2102]
  • [removal] Removal of deprecated code in filters, io, surface and visualization modules [#2077]
  • Suppress strict alias warning [#2072]
  • Suppress unused parameter warnings [#2067]
  • [abi] Removed unused member from ply_parser [#2066]
  • Suppress control reaches end of non-void function in io.h [#2057]
  • Modify STRICT_ALIGN because macro expansion w/defined is undefined [#2043]
  • Add necessary boost headers to pcl/io to build in CUDA mode [#2025]
  • Fix MSVC compile issue related with ssize_t [#2027]
  • Adds in-memory PCD serialization/deserialization; de-duplicates PCDReader::readHeader(). (take Fix compilation #2) [#1986]

libpcl_kdtree:

  • Consistent ptr typedefs for kd tree flann [#1607]

libpcl_keypoints:

  • Add TrajkovicKeypoint2D/3D to CMake build [#2179]

libpcl_ml:

  • Fix installation rules for ml module [#2192]

libpcl_octree:

  • octree_key.h suppress GCC 8 ignored-qualifier warning [#2405]
  • [api][new-feature] Implementation of the iterator 'OctreeLeafNodeBreadthIterator'. [#2204]
  • [abi][new-feature] Implementation of the iterator 'OctreeFixedDepthIterator'. [#1983]
  • Octree Iterator begin/end method and added tests [#2174]
  • [api] Remove parametrization of end iterators [#2168]
  • Fix docstrings in octree test [#2173]
  • [api] OctreeIterators special member revision [#2108]
  • Remove unused variable from octree_viewer [#2069]
  • Silence compile warning by removing superfluous call to std::max() [#2014]
  • [OCTREE] Add bounding box checks in isVoxelOccupiedAtPoint() and deleteVoxelAtPoint() [#1976]

libpcl_outofcore:

  • Explictly use mt19937 random generator for boost 1.67. [#2338]
  • Fixed queryBBIncludes_subsample [#1988]

libpcl_people:

libpcl_recognition:

  • Add some missing eigen alignment operators [#2433]
  • Setting the resolution of the occupancy grid [#2273]
  • Inline helper function gcCorrespSorter() [#2248]
  • Misleading indentation [#2034]

libpcl_registration:

  • Add some missing eigen alignment operators [#2433]
  • Remove explicit initialization of update_visualizer_ in Registration [#2423]
  • [removal] Remove registration module deprecated methods [#2076]
  • Suppress (maybe) uninitialized warning [#2073]
  • Remove unreachable code in DefaultConvergenceCriteria [#1967]
  • [abi] Added option to specify translation and rotation convergence deltas in ICP and NDT algorithms. [#1724]

libpcl_sample_consensus:

  • [api] Const-qualify most of the methods in SAC model classes [#2270]
  • [removal] Remove deprecated functions and variables from SAC module [#2071]

libpcl_search:

  • Correct testPoint for organized nearest neighbor search [#2198]

libpcl_segmentation:

  • Add some missing eigen alignment operators [#2433]
  • Add setter/getter for search method in ConditionalEuclideanClustering [#2437]
  • Increase threshold for expected value in test_non_linear [#2424]
  • Avoid infinite recursion in getPointCloudDifference [#2402]
  • Correct setting of is_dense flag in SegmentDifferences. Deprecate unused parameter in method. [#2380]
  • Dereference shared_ptr, fix for GCC8 [#2299]
  • [abi][deprecation] Provide proper EuclideanClusterComparator method depreciation. New Pragma macro. New Deprecated type. [#2096]
  • Removed normal related accessors and types from EuclideanClusterComparator [#1542]

libpcl_simulation:

  • [api] Add const qualifiers to RangeLikelihood getters. [#2411]
  • [api] Use GLuint rather than size_t to represent OpenGL indices. [#2238]
  • Support both RGB and RGBA colors in mesh loading [#2036]
  • [abi] Redo: Simulation: enable returning of organized point clouds [#1687]
  • Simulation: more access to camera parameters [#1650]

libpcl_surface:

  • Add some missing eigen alignment operators [#2433]
  • Make pcl::MovingLeastSquares thread-safe [#2418]
  • [api] Missing pcl::MovingLeastSquaresOMP declaration without /openmp [#2324]
  • Improved docstrings and error messages [#2300]
  • opennurbs: fix ON_Curve::EvaluatePoint calculation [#2185]
  • [removal] Removal of deprecated code in filters, io, surface and visualization modules [#2077]
  • Suppress (maybe) uninitialized warning [#2073]
  • Suppress sign compare warnings [#2068]
  • Fix incorrect Ptr/ConstPtr typedefs in MovingLeastSquaresOMP [#2055]
  • Replace float indices with Eigen Index [#2017]
  • [api][new-feature] Add new mls projection method [#1960]
  • Avoid phantom surfces in marching cubes hoppe [#1874]
  • [api] Add ability to cache mls results [#1952]

PCL Apps:

  • demean_cloud correct usage message. [#2443]
  • Do not use deprecated method in openni_mls_smoothing app [#2421]
  • add windows.h for includes GL/gl.h; handle cancel for denoiseWidget. [#2267]
  • Add missing dependecy to apps [#2251]
  • Suppress the final set of warnings in pcl apps [#2082]
  • [abi] More warning suppression in pcl apps [#2080]

PCL Docs:

  • Docstring typos' corrections. [#2449]
  • demean_cloud correct usage message. [#2443]
  • Set IMAGE_PATH explicitly in Doxygen config [#2442]
  • Switch to using client-based search in Doxygen [#2391]
  • [api] Misc. typos (cont.) [#2215]
  • doc: misc. typos [#2213]
  • Add url to API/ABI compatibity report [#2116]
  • Improve Doxygen comments for HistogramInterpolationMethod [#2111]
  • Update organized.h [#1965]
  • Typo [#1968]
  • Fixed spelling and grammar errors [#1959]
  • Fixed error in documentation. [#1957]

PCL Tutorials:

  • Add settings for hypothesis verification [#2274]
  • Fix ICP tutorial [#2244]
  • Fix error in example code for estimate set of surface for a subset of points in the input dataset [#2203]
  • Update message about legacy point cloud types in tutorial [#2175]
  • Add descriptor unpacking to GASD tutorial [#2167]
  • Fix convert to Eigen::Map<const Eigen::Vector3f> from Normal of pcl::PointXYZINormal [#2128]
  • Fix the tutorial qt_visualizer compilation issue: qt4 -> qt5. [#2051]
  • Fix several documentation typos [#2020]
  • Replace literal include of wrong CMakeLists file with correct script [#1971]
  • Update Ensenso tutorial for Ensenso X devices [#1933]

PCL Examples:

  • Suppress strict alias warning [#2072]
  • Suppress (maybe) uninitialized warning [#2073]
  • Fix CPC/LCCP segmentation examples for VTK 7.1 [#2063]

PCL Tests:

  • Increase threshold for expected value in test_non_linear [#2424]
  • Replace floating point numerals when using boost::posix_time. Fix compatibility with Boost 1.67. [#2422]
  • Cleanup and improve unit test coverage for transformPointCloud functions [#2245]
  • Fixes and new assertion macro in "pcl_tests.h" [#2237]
  • Add new gtest path [#1920]

PCL Tools:

  • Allow the pcl_uniform_sampling tool to deal with several formats (PCD, PLY and VTK) as input or output point cloud [#2348]
  • mesh_sampling tool: Add support for colors [#2257]
  • Error message on non-recognized feature names [#2178]
  • Suppress sign compare warnings [#2068]
  • [OCTREE] Compute accurately the centroids of octree in 'pcl_octree_viewer' [#1981]
  • [api][new-feature] Add new mls projection method [#1960]
  • [OCTREE] Fix pcl_octree_viewer [#1973]
  • [OCTREE] Remove a useless field in octree_viewer. [#1980]

CI:

  • Fix docs on Travis CI. [#2441]
  • Disable SSE flags in AppVeyor. [#2438]
  • Split (yet again) Travis test job into two and tweak timings in building apps [#2182]
  • [new-feature] Enable Global Tests on Windows CI [#2137]
  • Travis merge test jobs. Expose VTK include directory for the visualization module. [#2163]
  • Remove unnecessary PPAs and packages from Travis [#2153]
  • AppVeyor - Change to simple style of specify triplet [#2135]
  • Initial Appveyor CI integration [#2083]
  • Change Travis to use pip3 for installing sphinx [#2124]
  • [TRAVIS] Enable the build of apps. [#2012]
  • [TRAVIS] Enable the build of tools. [#2007]
  • Disable tools build in CI. [#2003]

I'm going to revise the new special sections to ensure we're consistent with the descriptions

@SergioRAgostinho
Copy link
Member Author

API changes:

Changes to the API which didn't went through the proper deprecation and removal cycle.

  • [simulation] Add const qualifiers to RangeLikelihood getters. [#2411]
  • [octree] Implementation of the iterator 'OctreeLeafNodeBreadthIterator'. [#2204]
  • [surface] Missing pcl::MovingLeastSquaresOMP declaration without /openmp [#2324]
  • [sample_consensus] Const-qualify most of the methods in SAC model classes [#2270]
  • [simulation] Use GLuint rather than size_t to represent OpenGL indices. [#2238]
  • [visualization] Fix access specifier in PointCloudColorHandlerRGBAField [#2226]
  • [docs] Misc. typos (cont.) [#2215]
  • [filters] Add const qualifiers to getters in filter_indices.h [#2193]
  • [octree] Remove parametrization of end iterators [#2168]
  • [octree] OctreeIterators special member revision [#2108]
  • [io] Add support pcl::PointXYZRGBA to pcl::VLPGrabber [#2102]
  • [surface][tools] Add new mls projection method [#1960]
  • [surface] Add ability to cache mls results [#1952]

These are the current (undeprecated) API changes. Here's my current proposal in terms of course of action.

Leave it as it is:

To discuss:

@SergioRAgostinho SergioRAgostinho mentioned this pull request Sep 26, 2018
3 tasks
@taketwo
Copy link
Member

taketwo commented Sep 27, 2018

I agree with your proposals in the "leave as it is" section. As for the discussion points:

  1. We do break API (at least in Const-qualify most of the methods in SAC model classes #2270) because these methods are virtual. Overriding methods in deriving classes have to be updated, otherwise they are not overriding anymore (const and non-const are different methods).
  2. Agree, changed the tag.
  3. Agree, though not entirely certain.
  4. I would leave it as it is.

@SergioRAgostinho
Copy link
Member Author

SergioRAgostinho commented Sep 28, 2018

1. Agreed. In the other cases they're not virtual so I will remove the tags from those.
3. Let's wing it.

I'll generate the final change log and push it here once I review the rest of the pending emails.

@SergioRAgostinho
Copy link
Member Author

Started going carefully through the abi/api reports and here's what it (additionally) complained about:

  1. [PARSE] Constness of the API #2224 added const qualifiers to a number of parameters in free functions. AFAIK they should implicitly converted.
  2. Improved docstrings and error messages #2300 we added a const qualifier to a string parameter of the ctors of FieldComparison and PackedRGBComparison. Same as above.
  3. Add support pcl::PointXYZRGBA to pcl::VLPGrabber. Deprecate rgb signatures. #2102 we have a hint why it was considered and API breakage. We renamed protected members. If people derive, which we allow it, their code might break. Propose to leave it as is.

Final change log coming now.

@SergioRAgostinho
Copy link
Member Author

I will remove the change log update commit from this PR and add it to #2489, so that it is up to date when I bump versions.

@SergioRAgostinho
Copy link
Member Author

Shall we merge this now or only after we implement the PR grouping strategy?

@taketwo
Copy link
Member

taketwo commented Oct 15, 2018

I don't have a strong opinion on this, both options are fine for me.

@SergioRAgostinho
Copy link
Member Author

Then merge it. I'm trying to write a blog post about this and that way I can link to the file on PCL's repo.

@taketwo taketwo merged commit a8bedab into PointCloudLibrary:master Oct 15, 2018
@SergioRAgostinho SergioRAgostinho deleted the change-log-tool branch November 8, 2018 14:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Change Log automation duplicate macro definition in instantiate.hpp header
2 participants