Quantcast

Eclipse plugin incremental analysis not working for team work

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Eclipse plugin incremental analysis not working for team work

ivano
There are 5 people working on the project and I've configured Jenkins to run a full analysis whenever a change is submitted, against his own P4 workspace.

I would expect that after the full analysis by Jenkins the eclipse plugin only scan newly edited file, but is instead running the full analysis every time.
To make it work properly I have to run an analysis via SonarRunner on my machine. However this fixes the problem only for me: my colleagues still see ALL the errors as new.

How does SonarQube decide if an issue is new or not? If people have to run a full analysis every time there is no point using the plugin.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Eclipse plugin incremental analysis not working for team work

Julien HENRY
Hi Ivano,

This may be a silly question but are you all (including Jenkins) sharing the same SQ server instance? Could you precise the version of SQ you are using?

And to answer your question a file is detected to be new when its hash is different compared to the latest analyzed file on server.

++

Julien

2014-09-26 16:24 GMT+02:00 ivano <[hidden email]>:
There are 5 people working on the project and I've configured Jenkins to run
a full analysis whenever a change is submitted, against his own P4
workspace.

I would expect that after the full analysis by Jenkins the eclipse plugin
only scan newly edited file, but is instead running the full analysis every
time.
To make it work properly I have to run an analysis via SonarRunner on my
machine. However this fixes the problem only for me: my colleagues still see
ALL the errors as new.

How does SonarQube decide if an issue is new or not? If people have to run a
full analysis every time there is no point using the plugin.




--
View this message in context: http://sonarqube.15.x6.nabble.com/Eclipse-plugin-incremental-analysis-not-working-for-team-work-tp5028570.html
Sent from the SonarQube Users mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Eclipse plugin incremental analysis not working for team work

ivano
Hi Julien,

This may be a silly question but are you all (including Jenkins) sharing
the same SQ server instance? Could you precise the version of SQ you are
using?
Yes, we all point to the same instance of SonarQube that is running on a separate linux machine. The version is 4.0 (we cannot upgrade because of some bugs on higher versions).


And to answer your question a file is detected to be new when its hash is
different compared to the latest analyzed file on server.
I know that. But against what is it checked? We develop on Windows machines and SonarQube is running on Linux. In order to make everything work I set up a P4 workspace folder on the Linux machine, which Jenkins updates whenever there is a change and runs SQ.
Will the files' hash be the same on Windows and Linux? Will it be the same on the different Windows machines?

For what I'm seeing I would say NO.


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Eclipse plugin incremental analysis not working for team work

Julien HENRY
Hi,
2014-09-29 17:31 GMT+02:00 ivano <[hidden email]>:
Will the files' hash be the same on Windows and Linux? Will it be the same
on the different Windows machines?

This use case should be supported. For example lines ends are normalized before computing the hash. This allows for example to not be impacted by git autocrlf settings (that will automatically checkout files with \n on Linux and \r\n on Windows).

Are you relying on default platform encoding to do analysis? I know another corner case where people are using ISO-8859-1 encoding instead of Windows-1252 on Windows and that can lead to different result if your code contains some non ascii chars.

But maybe there are other specific corner cases we are not aware of. Could you please take a same file from both your Linux workspace and your Windows workspace and send it to me (privately) so that I can do a comparison?

++

Julien
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Eclipse plugin incremental analysis not working for team work

Julien HENRY
Hi Ivano,

I've checked the 2 files that you sent to me privately. They indeed have different line ends but as I said this case is properly handled and in both case the file hash is 14aa4ac2a655a989484753cc88bf118b.

1) I forgot to ask you: are you using sonar.branch analysis parameter in one of you analysis. This is one of the common mistake. One analysis is performed with sonar.branch=xyz but not the other. As a result the two projects are considered as different and all files as new.

2) Can you double check that both analysis are correctly configured to use UTF-8 source encoding (send me logs of both analysis if you are not sure)?

3) Can you have a look in the DB to see if hash is correctly stored:
SELECT p.kee, d.snapshot_data FROM sonar.snapshot_data d, projects p where d.resource_id = p.id and data_type='file_hashes';
should return one entry per project. Look for your project and you should see something like:
<your project key> <semi-colon separated list of hash per file, including src/path/to/Basket.java=14aa4ac2a655a989484753cc88bf118b>
please check that hash is the same for you.

++

Julien


2014-09-30 10:22 GMT+02:00 Julien HENRY <[hidden email]>:
Hi,
2014-09-29 17:31 GMT+02:00 ivano <[hidden email]>:
Will the files' hash be the same on Windows and Linux? Will it be the same
on the different Windows machines?

This use case should be supported. For example lines ends are normalized before computing the hash. This allows for example to not be impacted by git autocrlf settings (that will automatically checkout files with \n on Linux and \r\n on Windows).

Are you relying on default platform encoding to do analysis? I know another corner case where people are using ISO-8859-1 encoding instead of Windows-1252 on Windows and that can lead to different result if your code contains some non ascii chars.

But maybe there are other specific corner cases we are not aware of. Could you please take a same file from both your Linux workspace and your Windows workspace and send it to me (privately) so that I can do a comparison?

++

Julien

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Eclipse plugin incremental analysis not working for team work

ivano
Hi Julien,


1) I forgot to ask you: are you using sonar.branch analysis parameter in
one of you analysis. This is one of the common mistake. One analysis is
performed with sonar.branch=xyz but not the other. As a result the two
projects are considered as different and all files as new.
There is no sonar.branch parameter in the configuration, so I suppose it's disabled.

2) Can you double check that both analysis are correctly configured to use
UTF-8 source encoding (send me logs of both analysis if you are not sure)?

3) Can you have a look in the DB to see if hash is correctly stored:
SELECT p.kee, d.snapshot_data FROM sonar.snapshot_data d, projects p where
d.resource_id = p.id and data_type='file_hashes';
should return one entry per project. Look for your project and you should
see something like:
<your project key> <semi-colon separated list of hash per file, including
src/path/to/Basket.java=14aa4ac2a655a989484753cc88bf118b>
please check that hash is the same for you.
I've run again the analysis with SonarRunner (Windows) and then Jenkins (Linux). In the logs I can see they're both using UTF-8, but the on the Linux machine the locale is "en_US" while on the Windows machines is "en_GB".
The file hash is different for the two runs: 752a9bb28e72d736b10efca96313b2ca and 76bcdede78bef40e6725af520f32b688 respectively.

How do I specify the locale to use?

Thanks
Ivano
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Eclipse plugin incremental analysis not working for team work

Julien HENRY
You can force locale using standards Java properties (passed through MAVEN_OPTS or SONAR_RUNNER_OPTS env variables). But I don't think this is related to locale.

2014-10-02 12:29 GMT+02:00 ivano <[hidden email]>:
Hi Julien,



> 1) I forgot to ask you: are you using sonar.branch analysis parameter in
> one of you analysis. This is one of the common mistake. One analysis is
> performed with sonar.branch=xyz but not the other. As a result the two
> projects are considered as different and all files as new.

There is no sonar.branch parameter in the configuration, so I suppose it's
disabled.


> 2) Can you double check that both analysis are correctly configured to use
> UTF-8 source encoding (send me logs of both analysis if you are not sure)?
>
> 3) Can you have a look in the DB to see if hash is correctly stored:
> SELECT p.kee, d.snapshot_data FROM sonar.snapshot_data d, projects p where
> d.resource_id = p.id and data_type='file_hashes';
> should return one entry per project. Look for your project and you should
> see something like:
> <your project key>
>
> <semi-colon separated list of hash per file, including
> src/path/to/Basket.java=14aa4ac2a655a989484753cc88bf118b>
> please check that hash is the same for you.

I've run again the analysis with SonarRunner (Windows) and then Jenkins
(Linux). In the logs I can see they're both using UTF-8, but the on the
Linux machine the locale is "en_US" while on the Windows machines is
"en_GB".
The file hash is different for the two runs:
752a9bb28e72d736b10efca96313b2ca and 76bcdede78bef40e6725af520f32b688
respectively.

How do I specify the locale to use?

Thanks
Ivano



--
View this message in context: http://sonarqube.15.x6.nabble.com/Eclipse-plugin-incremental-analysis-not-working-for-team-work-tp5028570p5028714.html
Sent from the SonarQube Users mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Eclipse plugin incremental analysis not working for team work

ivano
In fact, after forcing the locale to en_GB the hash codes are still different.
Who generates the hash? The Sonar server or SonarRunner?

How do you generate the hash?

I have compared the files manually and the only differences are the line-ends characters and the file size.

Julien HENRY wrote
You can force locale using standards Java properties (passed through
MAVEN_OPTS or SONAR_RUNNER_OPTS env variables). But I don't think this is
related to locale.

2014-10-02 12:29 GMT+02:00 ivano <[hidden email]>:

> Hi Julien,
>
>
>
> > 1) I forgot to ask you: are you using sonar.branch analysis parameter in
> > one of you analysis. This is one of the common mistake. One analysis is
> > performed with sonar.branch=xyz but not the other. As a result the two
> > projects are considered as different and all files as new.
>
> There is no sonar.branch parameter in the configuration, so I suppose it's
> disabled.
>
>
> > 2) Can you double check that both analysis are correctly configured to
> use
> > UTF-8 source encoding (send me logs of both analysis if you are not
> sure)?
> >
> > 3) Can you have a look in the DB to see if hash is correctly stored:
> > SELECT p.kee, d.snapshot_data FROM sonar.snapshot_data d, projects p
> where
> > d.resource_id = p.id and data_type='file_hashes';
> > should return one entry per project. Look for your project and you should
> > see something like:
> > <your project key>
> >
> > <semi-colon separated list of hash per file, including
> > src/path/to/Basket.java=14aa4ac2a655a989484753cc88bf118b>
> > please check that hash is the same for you.
>
> I've run again the analysis with SonarRunner (Windows) and then Jenkins
> (Linux). In the logs I can see they're both using UTF-8, but the on the
> Linux machine the locale is "en_US" while on the Windows machines is
> "en_GB".
> The file hash is different for the two runs:
> 752a9bb28e72d736b10efca96313b2ca and 76bcdede78bef40e6725af520f32b688
> respectively.
>
> How do I specify the locale to use?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Eclipse plugin incremental analysis not working for team work

Julien HENRY

2014-10-03 15:03 GMT+02:00 ivano <[hidden email]>:
Who generates the hash? The Sonar server or SonarRunner?

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Eclipse plugin incremental analysis not working for team work

ivano
I've run FileMetadata.read on both files and the hash are different: 21185076 and 11502424
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Eclipse plugin incremental analysis not working for team work

Julien HENRY
Are talking about the exact same files you sent to me? What is the JRE you are using?

2014-10-03 17:52 GMT+02:00 ivano <[hidden email]>:
I've run FileMetadata.read on both files and the hash are different: 21185076
and 11502424



--
View this message in context: http://sonarqube.15.x6.nabble.com/Eclipse-plugin-incremental-analysis-not-working-for-team-work-tp5028570p5028764.html
Sent from the SonarQube Users mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Eclipse plugin incremental analysis not working for team work

Julien HENRY
Hi Ivano,

Again, are you talking about the same files you sent to me and for which I get 14aa4ac2a655a989484753cc88bf118b as hash? It would be super strange that we don't have the same value at least on Linux.

++

Julien

2014-10-06 10:12 GMT+02:00 ivano <[hidden email]>:
Sorry Julien,

I printed the wrong values.
The hashes are actually
7168cd236cbfba35742b79bec39f2357  for the Windows file and
5e96ae54c0bd7aad3b269a5c510a7dc0 for the Linux file

I've added a counter in the algorithm for the NON-ignored characters and for
the Windows' file I get two more (3268 against 3266)




Julien HENRY wrote
> Are talking about the exact same files you sent to me? What is the JRE you
> are using?
>
> 2014-10-03 17:52 GMT+02:00 ivano &lt;

> ivano.vingiani@

> &gt;:
>
>> I've run FileMetadata.read on both files and the hash are different:
>> 21185076
>> and 11502424
>>
>>
>>
>> --
>> View this message in context:
>> http://sonarqube.15.x6.nabble.com/Eclipse-plugin-incremental-analysis-not-working-for-team-work-tp5028570p5028764.html
>> Sent from the SonarQube Users mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>     http://xircles.codehaus.org/manage_email
>>
>>
>>





--
View this message in context: http://sonarqube.15.x6.nabble.com/Eclipse-plugin-incremental-analysis-not-working-for-team-work-tp5028570p5028790.html
Sent from the SonarQube Users mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Eclipse plugin incremental analysis not working for team work

ivano
In reply to this post by Julien HENRY
Sorry Julien,

Eclipse added some characters to the files, that's why the hash were different.

The hash for both files is now 821f72d51524dfd37f00b3cd468ffb1c and I get the same value in the database.

That is getting weird. I'll try again the analysis from Eclipse and see if it's working now.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Eclipse plugin incremental analysis not working for team work

Julien HENRY
Maybe you have an Eclipse plugin that is doing some weird stuff? Let me know how it goes since it may be of big interest for other people.

++

2014-10-06 18:04 GMT+02:00 ivano <[hidden email]>:
Sorry Julien,

Eclipse added some characters to the files, that's why the hash were
different.

The hash for both files is now 821f72d51524dfd37f00b3cd468ffb1c and I get
the same value in the database.

That is getting weird. I'll try again the analysis from Eclipse and see if
it's working now.



--
View this message in context: http://sonarqube.15.x6.nabble.com/Eclipse-plugin-incremental-analysis-not-working-for-team-work-tp5028570p5028811.html
Sent from the SonarQube Users mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Eclipse plugin incremental analysis not working for team work

ivano
Julien,

I've just noticed that the path stored in the database is different:
wsc/selfcare/src/com/cerillion/wsc/ui/client/subpages/Basket.java=821f72d51524dfd37f00b3cd468ffb1c
src/com/cerillion/wsc/ui/client/subpages/Basket.java=821f72d51524dfd37f00b3cd468ffb1c

This is because on Jenkins executes the analysis from the workspace root, while Eclipse does it from the project's folder.

You think that could be the issue?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Eclipse plugin incremental analysis not working for team work

Julien HENRY
Hi Ivano,

Yes, that's an issue. Configuration of project should be the same on regular CI analysis and on Eclipse side so that SQ *knows* it is actually comparing the same file in incremental analysis. And because we don't support file move the path of files should exactly match. Else it is considered as new file and analyzed again.

++

2014-10-07 8:59 GMT+02:00 ivano <[hidden email]>:
Julien,

I've just noticed that the path stored in the database is different:
wsc/selfcare/src/com/cerillion/wsc/ui/client/subpages/Basket.java=821f72d51524dfd37f00b3cd468ffb1c
src/com/cerillion/wsc/ui/client/subpages/Basket.java=821f72d51524dfd37f00b3cd468ffb1c

This is because on Jenkins executes the analysis from the workspace root,
while Eclipse does it from the project's folder.

You think that could be the issue?



--
View this message in context: http://sonarqube.15.x6.nabble.com/Eclipse-plugin-incremental-analysis-not-working-for-team-work-tp5028570p5028844.html
Sent from the SonarQube Users mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Eclipse plugin incremental analysis not working for team work

ivano
Hi Julien,

I finally managed to fix the issue.
I've added
-Dsonar.projectBaseDir=<pathToProject>

and now the records in the database are the same from SonarRunner and from Jenkins.

Thanks for your help.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Eclipse plugin incremental analysis not working for team work

Julien HENRY
Hi Ivano,

Thanks for the feedback.

2014-10-14 12:07 GMT+02:00 ivano <[hidden email]>:
Hi Julien,

I finally managed to fix the issue.
I've added
-Dsonar.projectBaseDir=<pathToProject>

and now the records in the database are the same from SonarRunner and from
Jenkins.

Thanks for your help.



--
View this message in context: http://sonarqube.15.x6.nabble.com/Eclipse-plugin-incremental-analysis-not-working-for-team-work-tp5028570p5029072.html
Sent from the SonarQube Users mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Eclipse plugin incremental analysis not working for team work

Christoph Forster
This post has NOT been accepted by the mailing list yet.
Hi All,

as mentioned in http://sonarqube.15.x6.nabble.com/SQ-Eclipse-Plugin-always-shows-all-Issues-as-quot-new-quot-td5035082.html I run in the same Problem, but did not manage to fix it with the sonar.projectBaseDir Parameter.
I added it as JVM Argument and as "Preview Analysis Property" but the changed path did not take effect (within SQ Log the original Path is written).

Any  Ideas?
Regards,
Christoph
Loading...