Cross-project analysis enabled - Takes hours...Is there a better way?

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Cross-project analysis enabled - Takes hours...Is there a better way?

Kyle Smith
Hello Sonar Users,

I have noticed that on large projects where there are many thousands of lines of code that the log files for the scanner say, "INFO - Cross-project analysis enabled" - The log hangs here for hours and then fails.

Should I disable this feature, 'Cross-project analysis' or does this indicate something else I am not aware of?

Thank-you for reading this. Any and all help is appreciated.

Regards,

Reply | Threaded
Open this post in threaded view
|

Re: Cross-project analysis enabled - Takes hours...Is there a better way?

Lakshmi
Hi all,

Even I'm facing same problem for multi-language analysis in SQ 4.3.
I have very large code base (maintenance projects) which takes hours to complete. We haven't ever faced this problem while using Sonar 3.4.1.

Instance -
a) A project with 2 Lakh+ lines of code took 1 hour to complete build.
b) Another project with 9 Lakh+lines of code took 6 hours to complete.

Please note - I tried with H2 & MYSQL. In both databases, build is taking hours to complete.
Thanks,
Lakshmi
Reply | Threaded
Open this post in threaded view
|

Re: Cross-project analysis enabled - Takes hours...Is there a better way?

Kyle Smith
Interesting Lakshmi. I face this problem on version 3.7.1 of SonarQube with the open source non-oracle verison of MySQL that is for all intents and purposes MySQL called MariaDB.

This morning I noticed some Read timed out errors in the log files with some projects with hundreds of thousands of lines of code. These projects normally run without errors most of the time though. 0_o
Reply | Threaded
Open this post in threaded view
|

Re: Cross-project analysis enabled - Takes hours...Is there a better way?

Simon Brandhof
This "cross-project analysis" is about detection of code duplications between modules and projects.

The quick and dirty solution is to disable this feature :
http://docs.codehaus.org/display/SONAR/Duplications#Duplications-ActivatingCrossModuleandCrossProjectDuplicationDetection

Slowness can come from various parameters like too low amount of memory on big projects (fixed in 4.4, see http://jira.codehaus.org/browse/SONAR-5189) or slow bandwith between analyzer and database. Is that your case ? How many thousands of ncloc have your project ? How many projects do you have ? Which SonarQube version do you use ?

Regards



Simon BRANDHOF | SonarSource
http://twitter.com/SimonBrandhof


On 5 August 2014 16:11, What_Just_Happened <[hidden email]> wrote:
Interesting Lakshmi. I face this problem on version 3.7.1 of /SonarQube/ with
the open source non-oracle verison of MySQL that is for all intents and
purposes MySQL called MariaDB.

This morning I noticed some *Read timed out* errors in the log files with
some projects with hundreds of thousands of lines of code. These projects
normally run without errors most of the time though. 0_o



--
View this message in context: http://sonarqube.15.x6.nabble.com/Cross-project-analysis-enabled-Takes-hours-Is-there-a-better-way-tp5027117p5027155.html
Sent from the SonarQube Users mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



Reply | Threaded
Open this post in threaded view
|

Re: Cross-project analysis enabled - Takes hours...Is there a better way?

Kyle Smith
Hi Simon,

Let me answer your questions:

Slowness can come from various parameters like too low amount of memory on big projects (fixed in 4.4, see http://jira.codehaus.org/browse/SONAR-5189) or slow bandwith between analyzer and database. Is that your case ? 
From what I read this is not recommended, but the database and analyzer are on the same operating-system.

How many thousands of ncloc have your project ? 
Around 200,000

How many projects do you have ?
Around 60

 Which SonarQube version do you use ?
3.7.1

What are your thoughts on this given this information? I finally was able to complete the scan. I think it helped to set the sonar.dryRun.readTimeout attribute to about 20 minutes in milliseconds.

Regards,

Reply | Threaded
Open this post in threaded view
|

Re: Cross-project analysis enabled - Takes hours...Is there a better way?

Kyle Smith
Simon,

I forgot to mention the amount of RAM on the machine is 7.7GB total with .5GB free.

I am not sure if the free amount is the amount available all the time or when a scan is being run. This is not a lot of RAM if .5GB is the only amount available. The network is not particularly speedy with the web interface taking some time to load.

Regards,