[BATCH-2341] JdbcPagingItemReader does not work with DISTINCT in SQL
SERVER Created: 26/Jan/15 Updated: 26/Jan/15 Resolved: 26/Jan/15
Status:
Project:
Component/s:
Affects
Version/s:
Fix Version/s:
Resolved
Spring Batch
Core
3.0.3
Type:
Reporter:
Resolution:
Labels:
Remaining
Estimate:
Time Spent:
Original
Estimate:
Bug
Driss Amri
Works as Designed
None
Not Specified
None
Priority:
Assignee:
Votes:
Major
Michael Minella
0
Not Specified
Not Specified
Description
Using the JdbcPagingItemReader in combination with this PagingQueryProvider:
@Bean
public PagingQueryProvider queryProvider() {
SqlPagingQueryProviderFactoryBean factory = new SqlPagingQueryProviderFactoryBean();
factory.setDataSource(dataSource);
factory.setDatabaseType("SQLSERVER");
factory.setSelectClause("SELECT DISTINCT projectnummer");
factory.setFromClause("FROM project");
factory.setSortKey("projectnummer");
it will use the SqlServerPagingQueryProvider to generate a TOP query, but this will generate
bad syntax for SQL server:
org.springframework.jdbc.BadSqlGrammarException: StatementCallback; bad SQL grammar
[SELECT TOP 100 DISTINCT projectnummer FROM project ORDER BY projectnummer
ASC]; nested exception is com.microsoft.sqlserver.jdbc.SQLServerException: Incorrect syntax
near the keyword 'DISTINCT'.
Correct syntax would be
SELECT DISTINCT TOP 100 projectnummer FROM project
Is there any workaround for this?
Comments
Comment by Michael Minella [ 26/Jan/15 ]
This is an issue with using DISTINCT and TOP with SqlServer. In short, it looks like you need
to perform a bit of SQL hackery to get the two to play nice:
http://stackoverflow.com/questions/1925176/sql-server-2008-top-10-and-distinct-together
Comment by Driss Amri [ 26/Jan/15 ]
I'm not sure it is. If I do: "select DISTINCT TOP 10 projectnummer from project" I do get
distinct first TOP 10 unique results as expected.
As a workaround I changed the FROM clause to "from (select distinct projectnummer from
project) project" which does seem to do the trick.
Comment by Michael Minella [ 26/Jan/15 ]
I'm going to consider this closed unless I hear otherwise.
Comment by Michael Minella [ 26/Jan/15 ]
To further respond to https://twitter.com/drams88/status/559760829831450626, because of the
possibility of there being more than one distinct on a given query, we don't want to get into
parsing the SQL which addressing this would require. While pull requests are always welcome,
since there is a valid work around, I'm hesitant to dive into the complexities of addressing this
given that a work around is available.
Comment by Driss Amri [ 26/Jan/15 ]
These different ways of building a DISTINCT are not unique to Microsoft SQL server I would
think though? Shouldn't the "basic" version work where you use "SELECT DISTINCT FROM
TABLE" at least?
I don't have any experience with the code base so can't really judge if it's a lot of work to
support this, but after taking a quick look it doesn't seem like a straight forward change or else I
wouldn't mind giving it a go. It should appended differently in SqlPagingQueryUtils but this is
generic for all databases so this is probably not the way to go.
Generated at Wed Feb 10 04:17:11 UTC 2016 using JIRA 6.4.11#64026sha1:78f6ec473a3f058bd5d6c30e9319c7ab376bdb9c.