How third-party libraries change code analysis rules

Imagine you add a third-party library and suddenly some diagnostic rules of the static analyzer stop working. In this article, we'll look at one of the reasons why this can happen and offer some effective strategies for fixing the issue.

The issue description

Why some diagnostic rules of the static analyzer may disappear when one enables a third-party library? The single and obvious answer is that the library disables the rules on its own. Why? In fact, the library developers may disable the rules for their own project, without considering that it may affect all developers who use the library as well.

Background note on the mechanism of false positive suppression in PVS-Studio

Our product started as a tool for analyzing C and C++ code. I think it's no secret that the main issue with static analyzers is false positives, and PVS-Studio is no exception. To address this issue, we offer several methods of false positive suppression.

The first mechanism was introduced in PVS-Studio 3.40 (released on November 23, 2009). A user only needed to add a comment of the following type at the end of the code line where the analyzer issued a warning:

//-Vxxxx

The xxxx is the number of the diagnostic rule. Once the comment had been added, the next time the analysis was run, the warning for that line was filtered out of the resulting report. You can find a similar solution in other products such as Clang-Tidy (// NOLINT, // NOLINTNEXTLINE).

This mechanism proved to be very effective and everything worked perfectly fine. As time went on, users had mastered it. After a while we started getting feedback that the analyzer was issuing false positives for code generated when macros were expanded. To reduce the number of false positives when using macros, we came up with the following solution in PVS-Studio 4.13 (released on February 11, 2011):

//-V:MACRO_NAME:xxxx

MACRO_NAME is the name of the function-like macro and xxxx is the number of the diagnostic rule.

Unlike the previous approach, the new one didn't require binding to a line of source code. Where was the right place for the comment then? The best option would be to put it in a separate file of the analyzer settings. However, this feature appeared only in PVS-Studio 6.04 (released on May 16, 2016). Up to that point, there were two possible locations for the comment:

• directly in the compiled file being checked;
• in one of the header files that is included in the checked compiled file.

Let's remember this crucial moment, we'll come back to it.

Since then, the mechanism has undergone numerous enhancements:

• disabling language-specific analysis: //-V::C++, //-V::C#, ...
• disabling a group of diagnostic rules: //-V::GA, //-V::OP, ....
• excluding warnings of a certain level from the analysis results: //-V::number:level.
• excluding warnings by a substring in the message: //-V::number::{substring}.
• and others.

However, one thing remained unchanged: users could still write such comments both in header files and in compiled files. Unfortunately, the solution adopted 13 years ago carried a fatal error. While writing such a comment in a compiled file is relatively harmless, in the case of header files, it can very easily shoot you in the foot. Moreover, this thing is easy to overlook.

Note. The issue is specific to the C and C++ code analyzers. You can fine-tune other analyzers only using special files.

Real-world example

How can this shoot you in the feet? Let's look at an example from Unreal Engine. It contains an interesting file called MicrosoftPlatformCodeAnalysis.h, where various settings are specified:

The contents of MicrosoftPlatformCodeAnalysis.h

//PVS-Studio settings:
//-V::505,542,581,601,623,668,677,690,704,719,720,730,735,751,....
//-V:TRYCOMPRESSION:519,547
//-V:check(:501,547,560,605
//-V:checkSlow(:547
//-V:checkf(:510
//-V:checkfSlow(:547
//-V:dtAssert(:568
//-V:rcAssert(:568
//-V:GET_FUNCTION_NAME_CHECKED:521
//-V:ENABLE_TEXT_ERROR_CHECKING_RESULTS:560
//-V:ENABLE_LOC_TESTING:560,617
//-V:WITH_EDITOR:560
//-V:UE_LOG_ACTIVE:560
//-V:verify:501
//-V:%n:609
//-V:UE_BUILD_SHIPPING:501
//-V:WITH_EDITOR:501
//-V:TestTrueExpr:501
//-V:PLATFORM_:517,547
//-V:ensureMsgf:562
//-V:WindowsMinorVersion:547
//-V:Import.XObject:547,560
//-V:MotionControllerComponent:547,560
//-V:AddUninitialized(sizeof(void*)/:514
//-V:TestTrue:678
//-V:SetViewTarget:678
//-V:Slot:607
//-V:RESIDENCY_CHECK_RESULT:607
//-V:bHitTesting:581
//-V:OptionalType:580 
//-V:GetNextNode:681
//-V:ConvertToAbsolutePathForExternalAppFor:524
//-V:CopySingleValue:524
//-V:bTimeLimitReached:560
//-V:bRedirectionAllowed:560
//-V:NumFailures:560
//-V:bAllowInstantToolTips:560
//-V:bIsRealTime:560
//-V:Position:519
//-V:DynamicParameterValue[ParameterIndex]:557
//-V:ViewIndex:557
//-V:DeviceIndex:557
//-V:Interpolation:560
//-V:storePortals:560
//-V:bDefaultShouldBeMaximized:560
//-V:bAllowPerfHUD:560
//-V:bUseClientStorage:560
//-V:bCalculateThisMapping:560
//-V:bDebugSelectedTaskOnly:560
//-V:bDebugSelectedTaskOnly:560
//-V:bIsPreview:560
//-V:bSupportsFastClear:560
//-V:bUseAPILibaries:560
//-V:bUseCachedBlobs:560
//-V:bWireframe:560
//-V:Num():560
//-V:PLATFORM_MAC:560
//-V:Particle->Size.Z:570
//-V:ComponentMaskParameter:601
//-V:Format(:601
//-V:SelectedEmitter:519
//-V:MAX_VERTS_PER_POLY:512
//-V:127:547
//-V:0x7F:547
//-V:WARN_COLOR:547
//-V:<<:614
//-V:FT_LOAD_TARGET_NORMAL:616
//-V:OPENGL_PERFORMANCE_DATA_INVALID:564
//-V:HLSLCC_VersionMajor:616
//-V:bIgnoreFieldReferences:519
//-V:CachedQueryInstance:519
//-V:MeshContext:519
//-V:bAffectedByMarquee:519
//-V:CopyCompleteValueFromScriptVM:524
//-V:OnStopWatchingPin:524
//-V:GetMinChildNodes:524
//-V:FromWorldMatrix:524
//-V:RemoveSelectedActorsFromSelectedLayer_CanExecute:524
//-V:NotifyLevelRemovedFromWorld:524
//-V:SPAWN_INIT:595
//-V:BEGIN_UPDATE_LOOP:595
//-V:OPENGL_PERFORMANCE_DATA_INVALID:560
//-V:bSkipTranslationTrack:560
//-V:NumSelected>0:581
//-V:bTryPerTrackBitwiseCompression:581
//-V:DataStripped:581
//-V:FromInt:601
//-V:UE_CLOG(:501,560
//-V:UE_LOG(:501,510,560
//-V:UGL_REQUIRED_VOID:501
//-V:AnimScriptInstance:595
//-V:Driver:595
//-V:PSceneAsync->lockWrite:595
//-V:Context.World():595
//-V:UNIT_LOG:595
//-V:ensure(:595
//-V:ALLOCATE_VERTEX_DATA_TEMPLATE:501
//-V:UGL_REQUIRED:501
//-V:DEBUG_LOG_HTTP:523
//-V:GIsEditor:560
//-V:bHasEditorToken:560
//-V:GEventDrivenLoaderEnabled:501
//-V:WALK_TO_CHARACTER:519
//-V:IMPLEMENT_AI_INSTANT_TEST:773
//-V:ENABLE_VERIFY_GL:564
//-V:INC_MEMORY_STAT_BY:568
//-V:DEC_MEMORY_STAT_BY:568
//-V:Key():568
//-V:Modify:762
//-V:GetTransitionList:762
//-V:Execute:768
//-V:LAUNCHERSERVICES_SHAREABLEPROJECTPATHS:768
//-V:SELECT_STATIC_MESH_VERTEX_TYPE:622
//-V:GET_FUNCTION_NAME_CHECKED:685
//-V:This(:678
//-V:state->error:649
//-V:ProjModifiers:616
//-V:PERF_DETAILED_PER_CLASS_GC_STATS:686
//-V:FMath:656
//-V:->*:607
//-V:GENERATED_UCLASS_BODY:764
//-V:CalcSegmentCostOnPoly:764
//-V:DrawLine:764
//-V:vrapi_SubmitFrame:641
//-V:VertexData:773
//-V:Linker:678
//-V:self:678
//-V:AccumulateParentID:678
//-V:FindChar:679

The developers of Unreal Engine have done a great job of suppressing false positives in macros and other code, disabling a certain list of diagnostic rules for their own use:

//-V::505,542,581,601,623,668,677,690,704,719,720,730,735,751,....

A lot of companies use Unreal Engine to develop games. When they use PVS-Studio to analyze their own projects, the suppress file may be implicitly included into their compiled files. For example, it's enough to enable the basic CoreMinimal.h header file for this to happen.

This means the settings from the third-party component are now influencing the user's analysis. In case of Unreal Engine projects, most of the diagnostic rules will simply be disabled. As a result, devs won't get warnings from these rules in the report, even though they'd like to. This is a serious issue that needs to be addressed.

Solving the problem

Fixing false positives

Obviously, diagnostic rules in third-party libraries are disabled for a reason—primarily to avoid false positives. The root issue lies in their occurrence. Static analyzer developers can fix it by resolving the underlying issues. We, the PVS-Studio developers, work hard to eliminate false positives and enhance the quality of diagnostic rules.

Forbid reading analysis settings from source code

To avoid the issue, all the user has to do is ensure that the settings don't spread from the source code of third-party components, mainly from header files. However, such a fix would break backward compatibility. It also seems that library developers who use PVS-Studio might be reluctant to manually edit source code and move settings to separate files.

A possible solution is to develop a migrator that will automatically process source code and make the necessary changes. However, creating it requires careful thought to ensure that the context of a program isn't altered. For example, the migrator must be capable of distinguishing between regular comments in code and string literals that contain such comments.

Clearly, the task isn't easy, but perhaps we can come to such a solution in the future.

Ignore settings from third-party code

One possible solution is to ignore settings from the third-party code. We needed a feature that makes it possible to ignore disabled diagnostic rules from third-party libraries in our projects. Our original plan was to extend the functionality of the --exclude-path flag, which excludes specified files and directories from the analysis—by adding the option to prevent the settings from being applied. However, we abandoned this idea due to concerns about breaking backwards compatibility for users.

As a result, we decided to add a special flag into the C and C++ analyzer core and also into the pvs-studio-analyzer utility:

--analysis-paths mode=path

• mode is a set of the following values:
  • skip-analysis excludes specified files and directories from the analysis;
  • skip-settings ignores reading settings from specified files and directories;
  • skip combines the features of the skip-analysis and skip-settings modes.
• path represents files and directories to which the settings will be applied.

Here are some examples:

--analysis-paths skip-analysis=*/third-party/*
--analysis-paths skip-settings=*/third-party/*
--analysis-paths skip=*/third-party/*

We have also provided the capability to write this flag to the pvsconfig configuration files in the following way:

//V_ANALYSIS_PATHS mode=path

Here are the options of how you can use it:

//V_ANALYSIS_PATHS skip-analysis=*/third-party/*
//V_ANALYSIS_PATHS skip-settings=*/third-party/*
//V_ANALYSIS_PATHS skip=*/third-party/*

Using the ; character, you can set several values in one flag:

--analysis-paths skip-analysis=*/third-party/*;skip-settings=*/test/*

Or in pvsconfig:

//V_ANALYSIS_PATHS skip-analysis=*/third-party/*;skip-settings=*/test/*

Let's see how you can leverage the new mode with Unreal Engine. Two scenarios for analyzing Unreal Engine projects exist: you can use UnrealBuildTool or the compilation monitoring utility (CLMonitoring).

Analyzing via the UnrealBuildTool utility: for your convenience, UnrealBuildTool now automatically passes the --analysis-paths flag with the correct modes and paths to the analyzer if you build a project with the -StaticAnalyzerProjectOnly flag. This flag runs the user project analysis, ignoring the Unreal Engine core module. The feature will become available in PVS-Studio 7.34 and Unreal Engine 5.5.2.

Analyzing via monitoring: create a file that has the pvsconfig extension with the following configuration:

//V_ANALYSIS_PATHS skip-settings=*\UE*\Engine\Source\*

In the C and C++ Compiler Monitoring UI application, before you start the monitoring process, specify the path to the pvsconfig file in the input field:

If you're using the CLMonitoring console utility, pass the -c flag with the path to the pvsconfig file:

CLMonitor.exe monitor
%YOUR_BUILD_COMMAND%
CLMonitor.exe analyze -l "path/to/report.plog" ^
                      -c "path/to/settings.pvsconfig"

Conclusion

We've looked into an important issue related to the disappearance of PVS-Studio diagnostic rules when third-party libraries are enabled. In the past, this limitation prevented developers from detecting potential errors in their code, which negatively affected code quality in projects. It also posed challenges for newcomers exploring the analyzer, as not all diagnostic capabilities of PVS-Studio were fully accessible.

The introduction of new features to ignore settings from third-party code offers developers greater control over code analysis. We hope these enhancements will make the development process more efficient and secure.

Click here to try PVS-Studio for free.