Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add linux and mingw x64 target to the paging extension #4280

Conversation

chippmann
Copy link
Contributor

Resolves #4272

This adds the linuxX64 and mingwX64 targets to the paging extension.

Also the linking was done on each binary but is only necessary on the test binaries (as far as i can see). This PR only links the sqlite library if the binary is a test binary.


Note:
The version of paging-mpp in libs.versions.toml still needs updating once cashapp/multiplatform-paging#121 is in a release.

Until this has happened, this PR should probably remain a draft

**Note:**
The version of `paging-mpp` still needs updating once cashapp/multiplatform-paging#121 is in a release
Copy link
Collaborator

@AlecKazakova AlecKazakova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 will merge once the paging lib is released

@AlecKazakova
Copy link
Collaborator

(and tests pass)

@veyndan
Copy link
Collaborator

veyndan commented Jun 20, 2023

@chippmann Your PR is part of app.cash.paging:paging-common:3.1.1-0.3.0!

@hfhbd hfhbd marked this pull request as ready for review June 22, 2023 07:44
@hfhbd hfhbd enabled auto-merge (squash) June 22, 2023 07:45
@chippmann
Copy link
Contributor Author

The reason for the failing test seem to be an sqlite compiling error:

> Task :extensions:androidx-paging3:linuxX64Test FAILED
error while compiling: SELECT value
FROM (
  SELECT
    value,
    CASE
      WHEN (row_number() OVER(ORDER BY value ASC) - 1) % ? = 0 THEN 1
      WHEN value = ? THEN 1
      ELSE 0
    END page_boundary
  FROM testTable
  ORDER BY value ASC
)
WHERE page_boundary = 1;
near "OVER": syntax error | error code SQLITE_ERROR
co.touchlab.sqliter.interop.SQLiteExceptionErrorCode: error while compiling: SELECT value
FROM (
  SELECT
    value,
    CASE
      WHEN (row_number() OVER(ORDER BY value ASC) - 1) % ? = 0 THEN 1
      WHEN value = ? THEN 1
      ELSE 0
    END page_boundary
  FROM testTable
  ORDER BY value ASC
)
WHERE page_boundary = 1;
near "OVER": syntax error
    at 0   test.kexe                           0x4b4081           kfun:kotlin.Throwable#<init>(kotlin.String?){} + 113 
    at 1   test.kexe                           0x4adecd           kfun:kotlin.Exception#<init>(kotlin.String?){} + 109 
    at 2   test.kexe                           0x6625fc           kfun:co.touchlab.sqliter.interop.SQLiteException#<init>(kotlin.String;co.touchlab.sqliter.interop.SqliteDatabaseConfig){} + 124 
    at 3   test.kexe                           0x662733           kfun:co.touchlab.sqliter.interop.SQLiteExceptionErrorCode#<init>(kotlin.String;co.touchlab.sqliter.interop.SqliteDatabaseConfig;kotlin.Int){} + 243 
    at 4   test.kexe                           0x6682e8           kfun:co.touchlab.sqliter.interop.SqliteDatabase#prepareStatement(kotlin.String){}co.touchlab.sqliter.interop.SqliteStatement + 3256 
    at 5   test.kexe                           0x678269           kfun:co.touchlab.sqliter.native.NativeDatabaseConnection#createStatement(kotlin.String){}co.touchlab.sqliter.Statement + 281 
    at 6   test.kexe                           0x6537f4           kfun:co.touchlab.sqliter.concurrency.ConcurrentDatabaseConnection#createStatement(kotlin.String){}co.touchlab.sqliter.Statement + 532 
    at 7   test.kexe                           0x435c71           kfun:app.cash.sqldelight.driver.native.ThreadConnection#useStatement(kotlin.Int?;kotlin.String){}co.touchlab.sqliter.Statement + 753 
    at 8   test.kexe                           0x43449f           kfun:app.cash.sqldelight.driver.native.ConnectionWrapper.accessStatement$lambda$0#internal + 479 
    at 9   test.kexe                           0x434ce4           kfun:app.cash.sqldelight.driver.native.ConnectionWrapper.$accessStatement$lambda$0$FUNCTION_REFERENCE$14.invoke#internal + 132 
    at 10  test.kexe                           0x432126           kfun:app.cash.sqldelight.driver.native.NativeSqliteDriver#accessConnection(kotlin.Boolean;kotlin.Function1<app.cash.sqldelight.driver.native.ThreadConnection,0:0>){0§<kotlin.Any?>}0:0 + 630 
    at 11  test.kexe                           0x433df3           kfun:app.cash.sqldelight.driver.native.ConnectionWrapper.accessStatement#internal + 547 
    at 12  test.kexe                           0x434251           kfun:app.cash.sqldelight.driver.native.ConnectionWrapper#executeQuery(kotlin.Int?;kotlin.String;kotlin.Function1<app.cash.sqldelight.db.SqlCursor,app.cash.sqldelight.db.QueryResult<0:0>>;kotlin.Int;kotlin.Function1<app.cash.sqldelight.db.SqlPreparedStatement,kotlin.Unit>?){0§<kotlin.Any?>}app.cash.sqldelight.db.QueryResult<0:0> + 497 
    at 13  test.kexe                           0x45b122           kfun:app.cash.sqldelight.paging3.KeyedQueryPagingSourceTest.object-2.execute#internal + 418 
    at 14  test.kexe                           0x4269f1           kfun:app.cash.sqldelight.ExecutableQuery#executeAsList(){}kotlin.collections.List<1:0> + 209 
    at 15  test.kexe                           0x43d665           kfun:app.cash.sqldelight.paging3.KeyedQueryPagingSource.load$lambda$1$lambda$0#internal + 1045 
    at 16  test.kexe                           0x43e6e2           kfun:app.cash.sqldelight.paging3.KeyedQueryPagingSource.$load$lambda$1$lambda$0$FUNCTION_REFERENCE$0.invoke#internal + 114 
    at 17  test.kexe                           0x429f1d           kfun:app.cash.sqldelight.TransacterImpl.transactionWithWrapper#internal + 1277 
    at 18  test.kexe                           0x4299e3           kfun:app.cash.sqldelight.TransacterImpl#transactionWithResult(kotlin.Boolean;kotlin.Function1<app.cash.sqldelight.TransactionWithReturn<0:0>,0:0>){0§<kotlin.Any?>}0:0 + 131 
    at 19  test.kexe                           0x428ac7           kfun:app.cash.sqldelight.Transacter#transactionWithResult$default(kotlin.Boolean;kotlin.Function1<app.cash.sqldelight.TransactionWithReturn<0:0>,0:0>;kotlin.Int){0§<kotlin.Any?>}0:0 + 279 
    at 20  test.kexe                           0x43e0e7           kfun:app.cash.sqldelight.paging3.KeyedQueryPagingSource.$load$lambda$1COROUTINE$0.invokeSuspend#internal + 599 
    at 21  test.kexe                           0x43e619           kfun:app.cash.sqldelight.paging3.KeyedQueryPagingSource.$load$lambda$1COROUTINE$0.invoke#internal + 297 
    at 22  test.kexe                           0x72757e           kfun:kotlinx.coroutines.intrinsics#startUndispatchedOrReturn__at__kotlinx.coroutines.internal.ScopeCoroutine<0:0>(0:1;kotlin.coroutines.SuspendFunction1<0:1,0:0>){0§<kotlin.Any?>;1§<kotlin.Any?>}kotlin.Any? + 910 
    at 23  test.kexe                           0x6813b4           kfun:kotlinx.coroutines#withContext#suspend(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.CoroutineScope,0:0>;kotlin.coroutines.Continuation<0:0>){0§<kotlin.Any?>}kotlin.Any? + 996 
    at 24  test.kexe                           0x43d20c           kfun:app.cash.sqldelight.paging3.KeyedQueryPagingSource#load#suspend(androidx.paging.PagingSource.LoadParams<1:0>;kotlin.coroutines.Continuation<androidx.paging.PagingSource.LoadResult<1:0,1:1>>){}kotlin.Any + 332 
    at 25  test.kexe                           0x45617a           kfun:app.cash.sqldelight.paging3.KeyedQueryPagingSourceTest.$aligned_page_exhaustion_gives_correct_results$lambda$0COROUTINE$0.invokeSuspend#internal + 1386 
    at 26  test.kexe                           0x4568a9           kfun:app.cash.sqldelight.paging3.KeyedQueryPagingSourceTest.$aligned_page_exhaustion_gives_correct_results$lambda$0COROUTINE$0.invoke#internal + 297 
    at 27  test.kexe                           0x48181b           kfun:app.cash.sqldelight.paging3.$runDbTest$lambda$0COROUTINE$0.invokeSuspend#internal + 1963 
    at 28  test.kexe                           0x481e79           kfun:app.cash.sqldelight.paging3.$runDbTest$lambda$0COROUTINE$0.invoke#internal + 297 
    at 29  test.kexe                           0x7daac6           kfun:kotlinx.coroutines.test.$runTest$lambda$6$lambda$0COROUTINE$1.invokeSuspend#internal + 886 
    at 30  test.kexe                           0x4b80b4           kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} + 916 
    at 31  test.kexe                           0x7180e2           kfun:kotlinx.coroutines.DispatchedTask#run(){} + 3490 
    at 32  test.kexe                           0x7e5d67           kfun:kotlinx.coroutines.test.TestDispatcher#processEvent(kotlin.Any){} + 423 
    at 33  test.kexe                           0x7e2f58           kfun:kotlinx.coroutines.test.TestCoroutineScheduler#tryRunNextTaskUnless(kotlin.Function0<kotlin.Boolean>){}kotlin.Boolean + 1352 
    at 34  test.kexe                           0x7db47c           kfun:kotlinx.coroutines.test.$runTest$lambda$6$lambda$2COROUTINE$2.invokeSuspend#internal + 828 
    at 35  test.kexe                           0x4b80b4           kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} + 916 
    at 36  test.kexe                           0x7180e2           kfun:kotlinx.coroutines.DispatchedTask#run(){} + 3490 
    at 37  test.kexe                           0x696d2f           kfun:kotlinx.coroutines.EventLoopImplBase#processNextEvent(){}kotlin.Long + 1583 
    at 38  test.kexe                           0x7464c4           kfun:kotlinx.coroutines.BlockingCoroutine.joinBlocking#internal + 660 
    at 39  test.kexe                           0x744ae8           kfun:kotlinx.coroutines#runBlocking(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.CoroutineScope,0:0>){0§<kotlin.Any?>}0:0 + 2120 
    at 40  test.kexe                           0x744daa           kfun:kotlinx.coroutines#runBlocking$default(kotlin.coroutines.CoroutineContext?;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.CoroutineScope,0:0>;kotlin.Int){0§<kotlin.Any?>}0:0 + 298 
    at 41  test.kexe                           0x7eef2a           kfun:kotlinx.coroutines.test#createTestResult(kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.CoroutineScope,kotlin.Unit>){} + 186 
    at 42  test.kexe                           0x7d9da9           kfun:kotlinx.coroutines.test#runTest__at__kotlinx.coroutines.test.TestScope(kotlin.time.Duration;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.test.TestScope,kotlin.Unit>){} + 409 
    at 43  test.kexe                           0x7d98bc           kfun:kotlinx.coroutines.test#runTest(kotlin.coroutines.CoroutineContext;kotlin.time.Duration;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.test.TestScope,kotlin.Unit>){} + 844 
    at 44  test.kexe                           0x7d9bdb           kfun:kotlinx.coroutines.test#runTest$default(kotlin.coroutines.CoroutineContext?;kotlin.time.Duration?;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.test.TestScope,kotlin.Unit>;kotlin.Int){} + 587 
    at 45  test.kexe                           0x480e9a           kfun:app.cash.sqldelight.paging3#runDbTest__at__app.cash.sqldelight.paging3.DbTest(kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.test.TestScope,kotlin.Unit>){} + 218 
    at 46  test.kexe                           0x454fe1           kfun:app.cash.sqldelight.paging3.KeyedQueryPagingSourceTest#aligned_page_exhaustion_gives_correct_results(){} + 177 
    at 47  test.kexe                           0x45d0ba           kfun:app.cash.sqldelight.paging3.$KeyedQueryPagingSourceTest$test$0.$aligned_page_exhaustion_gives_correct_results$FUNCTION_REFERENCE$7.invoke#internal + 74 
    at 48  test.kexe                           0x45d2d3           kfun:app.cash.sqldelight.paging3.$KeyedQueryPagingSourceTest$test$0.$aligned_page_exhaustion_gives_correct_results$FUNCTION_REFERENCE$7.$<bridge-UNNN>invoke(-1:0){}#internal + 99 
    at 49  test.kexe                           0x4efb6c           kfun:kotlin.native.internal.test.BaseClassSuite.TestCase#run(){} + 1740 
    at 50  test.kexe                           0x4e574d           kfun:kotlin.native.internal.test.TestRunner.run#internal + 7645 
    at 51  test.kexe                           0x4e7777           kfun:kotlin.native.internal.test.TestRunner.runIteration#internal + 4679 
    at 52  test.kexe                           0x4e88e3           kfun:kotlin.native.internal.test.TestRunner#run(){}kotlin.Int + 1443 
    at 53  test.kexe                           0x4d965f           kfun:kotlin.native.internal.test#testLauncherEntryPoint(kotlin.Array<kotlin.String>){}kotlin.Int + 255 
    at 54  test.kexe                           0x4d96fc           kfun:kotlin.native.internal.test#main(kotlin.Array<kotlin.String>){} + 76 
    at 55  test.kexe                           0x483415           Konan_start + 133 
    at 56  test.kexe                           0x48407a           Init_and_run_start + 122 
    at 57  libc.so.6                           0x7f3c5e22284f     0x0 + 139897254062159 
    at 58  libc.so.6                           0x7f3c5e222909     __libc_start_main + 137 
    at 59  test.kexe                           0x4264c2           0x0 + 4351170

I saw that you have a static sqlite.a library which you extract from a ubuntu 14.04 docker image https://github.com/cashapp/sqldelight/blob/master/libs/linux/Dockerfile
AFAIK the sqlite version in ubuntu 14.04 is 3.8.2 (https://launchpad.net/ubuntu/trusty/+source/sqlite3) which does not support window functions yet (supported since 3.25.0 https://www.sqlite.org/windowfunctions.html).

What was the reason to extract the sqlite lib from that ubuntu version and what do you think about using a sqlite amalgamation (https://www.sqlite.org/amalgamation.html) for the linux tests instead? That one could just be used with a def file instead.

One example would be (taken from one of my native projects using sqldelight on linuxX64):

# sqlite3 amalgamation (https://www.sqlite.org/amalgamation.html)
# only used for linux targets to prevent linking issues on newer linux distros with too recent libraries installed (like glibc)
# see: https://github.com/touchlab/SQLiter/pull/38#issuecomment-867171789

package = co.touchlab.sqliter.sqlite3

---
// the content of the amalgamation file goes here (omitted here as it's pretty big)

@hfhbd
Copy link
Collaborator

hfhbd commented Jun 23, 2023

@chippmann I copied the sqlite.a from 14.04 because this version was compiled with glibc 2.19, the same version used by Kotlin native compiler: #4040 (comment)

If switching to the armageddon file works, I am open.

auto-merge was automatically disabled June 23, 2023 09:23

Head branch was pushed to by a user without write access

@chippmann
Copy link
Contributor Author

@hfhbd Could you rerun the tests please (I do not have the necessary rights to do so).
The last failure was of a 500 request error while trying to download nodejs:
Could not HEAD 'https://nodejs.org/dist/v18.12.1/node-v18.12.1-linux-x64.tar.gz'. Received status code 500 from server: Internal Server Error and has most certainly nothing to do with these changes

@hfhbd hfhbd merged commit e799a73 into cashapp:master Jun 23, 2023
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add linux targets for app.cash.sqldelight:androidx-paging3-extensions
4 participants