Skip to content

Fineract.dev Demo Server Continous Deployment (CD) is broken since ~2021-09-21 #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
vorburger opened this issue Dec 28, 2021 · 11 comments
Assignees

Comments

@vorburger
Copy link
Owner

https://demo.fineract.dev/fineract-provider/actuator/info currently shows:

{
  "git":{
    "branch":"develop",
    "commit":{
      "time":"2021-09-21T13:34+0000",
      "message":{
        "full":"Feat: Repayment With Post Dated Checks\n",
        "short":"Feat: Repayment With Post Dated Checks"
      },
      "id":{
        "describe":"",
        "abbrev":"07f395e",
        "full":"07f395ef7f25fd8b5b3199c4476058a8afcc959e"
      },
      "user":{
        "email":"[email protected]",
        "name":"BLasan"
      }
    },
    "build":{
      "version":"unspecified",
      "user":{
        "name":"",
        "email":""
      },
      "host":"62eb7e3f6d5a"
    },
    "dirty":"true",
    "tags":"",
    "total":{
      "commit":{
        "count":"1"
      }
    },
    "closest":{
      "tag":{
        "commit":{
          "count":""
        },
        "name":""
      }
    },
    "remote":{
      "origin":{
        "url":"https://github.com/apache/fineract.git"
      }
    }
  }
}

This doesn't look right, given https://github.com/apache/fineract/commits/develop.

I'd like to fix this so that apache/fineract#2001 can be used for #5.

@ptuomola @awasum @vidakovic FYI

@vorburger
Copy link
Owner Author

It has been failing since 22.09.2021 due to this error:

Updating service [demo] (this may take several minutes)...
................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................failed.
ERROR: (gcloud.app.deploy) Error Response: [9] An internal error occurred while processing task /app-engine-flex/flex_await_healthy/flex_await_healthy>2021-10-20T03:34:36.804Z42410.wa.2: 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1307)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1227)
	at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:884)
	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:788)
	... 83 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'customAuthenticationProvider' defined in URL [jar:file:/app/fineract-provider.jar!/BOOT-INF/classes!/META-INF/spring/securityContext.xml]: Cannot resolve reference to bean 'userDetailsService' while setting bean property 'userDetailsService'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'userDetailsService': Unsatisfied dependency expressed through field 'platformUserRepository'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'appUserRepository' defined in org.apache.fineract.useradministration.domain.AppUserRepository defined in : Cannot create inner bean '(inner bean)#208205ed' of type [org.springframework.orm.jpa.SharedEntityManagerCreator] while setting bean property 'entityManager'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name '(inner bean)#208205ed': Cannot resolve reference to bean 'entityManagerFactory' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'tenantDatabaseUpgradeService': Invocation of init method failed; nested exception is org.flywaydb.core.api.FlywayException: Migration V370__message_gateway_hook_template.sql failed
--------------------------------------------------------
SQL State  : 21000
Error Code : 1242
Message    : Subquery returns more than 1 row
Location   : sql/migrations/core_db/V370__message_gateway_hook_template.sql (/app/file:/app/fineract-provider.jar!/BOOT-INF/classes!/sql/migrations/core_db/V370__message_gateway_hook_template.sql)
Line       : 25
Statement  : INSERT INTO m_hook_schema
(hook_template_id, field_type, field_name, placeholder, optional)
VALUES( (select id from m_hook_templates  where name = 'Message Gateway'), 'string', 'SMS Provider Id', NULL, 0)
; for Tenant DB URL: jdbc:mysql://10.103.176.3:3306/fineract_default, username: root
	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:342)
	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:113)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1697)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1442)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:516)
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:324)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:330)
	... 101 common frames omitted
Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'userDetailsService': Unsatisfied dependency expressed through field 'platformUserRepository'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'appUserRepository' defined in org.apache.fineract.useradministration.domain.AppUserRepository defined in : Cannot create inner bean '(inner bean)#208205ed' of type [org.springframework.orm.jpa.SharedEntityManagerCreator] while setting bean property 'entityManager'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name '(inner bean)#208205ed': Cannot resolve reference to bean 'entityManagerFactory' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'tenantDatabaseUpgradeService': Invocation of init method failed; nested exception is org.flywaydb.core.api.FlywayException: Migration V370__message_gateway_hook_template.sql failed
--------------------------------------------------------
SQL State  : 21000
Error Code : 1242
Message    : Subquery returns more than 1 row
Location   : sql/migrations/core_db/V370__message_gateway_hook_template.sql (/app/file:/app/fineract-provider.jar!/BOOT-INF/classes!/sql/migrations/core_db/V370__message_gateway_hook_template.sql)
Line       : 25
Statement  : INSERT INTO m_hook_schema
(hook_template_id, field_type, field_name, placeholder, optional)
VALUES( (select id from m_hook_templates  where name = 'Message Gateway'), 'string', 'SMS Provider Id', NULL, 0)
; for Tenant DB URL: jdbc:mysql://10.103.176.3:3306/fineract_default, username: root
	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:643)
	at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:130)
	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:399)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1420)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:516)
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:324)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:330)
	... 111 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'appUserRepository' defined in org.apache.fineract.useradministration.domain.AppUserRepository defined in : Cannot create inner bean '(inner bean)#208205ed' of type [org.springframework.orm.jpa.SharedEntityManagerCreator] while setting bean property 'entityManager'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name '(inner bean)#208205ed': Cannot resolve reference to bean 'entityManagerFactory' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'tenantDatabaseUpgradeService': Invocation of init method failed; nested exception is org.flywaydb.core.api.FlywayException: Migration V370__message_gateway_hook_template.sql failed
--------------------------------------------------------
SQL State  : 21000
Error Code : 1242
Message    : Subquery returns more than 1 row
Location   : sql/migrations/core_db/V370__message_gateway_hook_template.sql (/app/file:/app/fineract-provider.jar!/BOOT-INF/classes!/sql/migrations/core_db/V370__message_gateway_hook_template.sql)
Line       : 25
Statement  : INSERT INTO m_hook_schema
(hook_template_id, field_type, field_name, placeholder, optional)
VALUES( (select id from m_hook_templates  where name = 'Message Gateway'), 'string', 'SMS Provider Id', NULL, 0)
; for Tenant DB URL: jdbc:mysql://10.103.176.3:3306/fineract_default, username: root
	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:389)
	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:134)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1697)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1442)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:516)
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:324)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
	at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1307)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1227)
	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:640)
	... 121 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name '(inner bean)#208205ed': Cannot resolve reference to bean 'entityManagerFactory' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'tenantDatabaseUpgradeService': Invocation of init method failed; nested exception is org.flywaydb.core.api.FlywayException: Migration V370__message_gateway_hook_template.sql failed
--------------------------------------------------------
SQL State  : 21000
Error Code : 1242
Message    : Subquery returns more than 1 row
Location   : sql/migrations/core_db/V370__message_gateway_hook_template.sql (/app/file:/app/fineract-provider.jar!/BOOT-INF/classes!/sql/migrations/core_db/V370__message_gateway_hook_template.sql)
Line       : 25
Statement  : INSERT INTO m_hook_schema
(hook_template_id, field_type, field_name, placeholder, optional)
VALUES( (select id from m_hook_templates  where name = 'Message Gateway'), 'string', 'SMS Provider Id', NULL, 0)
; for Tenant DB URL: jdbc:mysql://10.103.176.3:3306/fineract_default, username: root
	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:342)
	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:113)
	at org.springframework.beans.factory.support.ConstructorResolver.resolveConstructorArguments(ConstructorResolver.java:690)
	at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:507)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1336)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1176)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:556)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:516)
	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:374)
	... 134 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'tenantDatabaseUpgradeService': Invocation of init method failed; nested exception is org.flywaydb.core.api.FlywayException: Migration V370__message_gateway_hook_template.sql failed
--------------------------------------------------------
SQL State  : 21000
Error Code : 1242
Message    : Subquery returns more than 1 row
Location   : sql/migrations/core_db/V370__message_gateway_hook_template.sql (/app/file:/app/fineract-provider.jar!/BOOT-INF/classes!/sql/migrations/core_db/V370__message_gateway_hook_template.sql)
Line       : 25
Statement  : INSERT INTO m_hook_schema
(hook_template_id, field_type, field_name, placeholder, optional)
VALUES( (select id from m_hook_templates  where name = 'Message Gateway'), 'string', 'SMS Provider Id', NULL, 0)
; for Tenant DB URL: jdbc:mysql://10.103.176.3:3306/fineract_default, username: root
	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:160)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:415)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1786)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:594)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:516)
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:324)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:311)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:330)
	... 142 common frames omitted
Caused by: org.flywaydb.core.api.FlywayException: Migration V370__message_gateway_hook_template.sql failed
--------------------------------------------------------
SQL State  : 21000
Error Code : 1242
Message    : Subquery returns more than 1 row
Location   : sql/migrations/core_db/V370__message_gateway_hook_template.sql (/app/file:/app/fineract-provider.jar!/BOOT-INF/classes!/sql/migrations/core_db/V370__message_gateway_hook_template.sql)
Line       : 25
Statement  : INSERT INTO m_hook_schema
(hook_template_id, field_type, field_name, placeholder, optional)
VALUES( (select id from m_hook_templates  where name = 'Message Gateway'), 'string', 'SMS Provider Id', NULL, 0)
; for Tenant DB URL: jdbc:mysql://10.103.176.3:3306/fineract_default, username: root
	at org.apache.fineract.infrastructure.core.service.TenantDatabaseUpgradeService.upgradeAllTenants(TenantDatabaseUpgradeService.java:89)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:389)
	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:333)
	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:157)
	... 153 common frames omitted
Caused by: org.flywaydb.core.internal.command.DbMigrate$FlywayMigrateException: Migration V370__message_gateway_hook_template.sql failed
--------------------------------------------------------
SQL State  : 21000
Error Code : 1242
Message    : Subquery returns more than 1 row
Location   : sql/migrations/core_db/V370__message_gateway_hook_template.sql (/app/file:/app/fineract-provider.jar!/BOOT-INF/classes!/sql/migrations/core_db/V370__message_gateway_hook_template.sql)
Line       : 25
Statement  : INSERT INTO m_hook_schema
(hook_template_id, field_type, field_name, placeholder, optional)
VALUES( (select id from m_hook_templates  where name = 'Message Gateway'), 'string', 'SMS Provider Id', NULL, 0)

	at org.flywaydb.core.internal.command.DbMigrate.doMigrateGroup(DbMigrate.java:378)
	at org.flywaydb.core.internal.command.DbMigrate.lambda$applyMigrations$1(DbMigrate.java:271)
	at org.flywaydb.core.internal.jdbc.TransactionalExecutionTemplate.execute(TransactionalExecutionTemplate.java:66)
	at org.flywaydb.core.internal.command.DbMigrate.applyMigrations(DbMigrate.java:270)
	at org.flywaydb.core.internal.command.DbMigrate.migrateGroup(DbMigrate.java:243)
	at org.flywaydb.core.internal.command.DbMigrate.lambda$migrateAll$0(DbMigrate.java:141)
	at org.flywaydb.core.internal.database.mysql.MySQLNamedLockTemplate.execute(MySQLNamedLockTemplate.java:60)
	at org.flywaydb.core.internal.database.mysql.MySQLConnection.lock(MySQLConnection.java:154)
	at org.flywaydb.core.internal.schemahistory.JdbcTableSchemaHistory.lock(JdbcTableSchemaHistory.java:141)
	at org.flywaydb.core.internal.command.DbMigrate.migrateAll(DbMigrate.java:141)
	at org.flywaydb.core.internal.command.DbMigrate.migrate(DbMigrate.java:101)
	at org.flywaydb.core.Flyway$1.execute(Flyway.java:219)
	at org.flywaydb.core.Flyway$1.execute(Flyway.java:170)
	at org.flywaydb.core.Flyway.execute(Flyway.java:586)
	at org.flywaydb.core.Flyway.migrate(Flyway.java:170)
	at org.apache.fineract.infrastructure.core.service.TenantDatabaseUpgradeService.upgradeAllTenants(TenantDatabaseUpgradeService.java:85)
	... 160 common frames omitted
Caused by: org.flywaydb.core.internal.sqlscript.FlywaySqlScriptException: Migration V370__message_gateway_hook_template.sql failed
--------------------------------------------------------
SQL State  : 21000
Error Code : 1242
Message    : Subquery returns more than 1 row
Location   : sql/migrations/core_db/V370__message_gateway_hook_template.sql (/app/file:/app/fineract-provider.jar!/BOOT-INF/classes!/sql/migrations/core_db/V370__message_gateway_hook_template.sql)
Line       : 25
Statement  : INSERT INTO m_hook_schema
(hook_template_id, field_type, field_name, placeholder, optional)
VALUES( (select id from m_hook_templates  where name = 'Message Gateway'), 'string', 'SMS Provider Id', NULL, 0)

	at org.flywaydb.core.internal.sqlscript.DefaultSqlScriptExecutor.handleException(DefaultSqlScriptExecutor.java:277)
	at org.flywaydb.core.internal.sqlscript.DefaultSqlScriptExecutor.executeStatement(DefaultSqlScriptExecutor.java:224)
	at org.flywaydb.core.internal.sqlscript.DefaultSqlScriptExecutor.execute(DefaultSqlScriptExecutor.java:128)
	at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor.executeOnce(SqlMigrationExecutor.java:78)
	at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor.lambda$execute$0(SqlMigrationExecutor.java:67)
	at org.flywaydb.core.internal.database.DefaultExecutionStrategy.execute(DefaultExecutionStrategy.java:27)
	at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor.execute(SqlMigrationExecutor.java:66)
	at org.flywaydb.core.internal.command.DbMigrate.doMigrateGroup(DbMigrate.java:370)
	... 175 common frames omitted
Caused by: java.sql.SQLException: Subquery returns more than 1 row
	at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:129)
	at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
	at com.mysql.cj.jdbc.StatementImpl.executeInternal(StatementImpl.java:762)
	at com.mysql.cj.jdbc.StatementImpl.execute(StatementImpl.java:646)
	at org.flywaydb.core.internal.jdbc.JdbcTemplate.executeStatement(JdbcTemplate.java:241)
	at org.flywaydb.core.internal.sqlscript.ParsedSqlStatement.execute(ParsedSqlStatement.java:111)
	at org.flywaydb.core.internal.sqlscript.DefaultSqlScriptExecutor.executeStatement(DefaultSqlScriptExecutor.java:212)
	... 181 common frames omitted

and some time over the weeks this error disappeared, and it now fails with some cryptic Google Cloud error which I do not yet understand.

@vorburger
Copy link
Owner Author

it now fails with some cryptic Google Cloud error which I do not yet understand.

OK, so that Flyway related problem shown above is indeed not the cause. (Or perhaps there is another problem BEFORE that problem and I'll see that again later - I'll have "unroll" this step by step.)

What's happening is that the internal liveness check fails - new versions never come alive on demo.fineract.dev - and the rollout is thus aborted, and we're stuck on the old version from September (which is nice and better than deploying a broken new version, isn't it?).

After a bit of digging (phew), I've found that the root cause of it is that new https://github.com/apache/fineract/blob/develop/entrypoint.sh which @IOhacker introduced in apache/fineract#1930 for https://issues.apache.org/jira/browse/FINERACT-1416. That, as-is currently, because the Dockerfile has an ENTRYPOINT /entrypoint.sh, makes a hard-coded assumption that anyone using the Fineract container anywhere uses fineractmysql 3306 as the hostname and port of the DB. And that... is a Bad Idea(TM)(R)... 😈 because I bet in some places incl. e.g. on Fineract.dev, the (Google Cloud SQL) DB just so happens to not be reachable by fineractmysql hostname like that!

TL;DR That (new) entrypoint.sh script really must respect those Fineract official DB configuration parameters, as seen e.g. in https://github.com/apache/fineract/blob/03a8a9858d1c3db703675ebaeb0828863d8b35de/docker-compose.yml and https://github.com/apache/fineract/blob/03a8a9858d1c3db703675ebaeb0828863d8b35de/kubernetes/fineract-server-deployment.yml. I'll raise a new JIRA about it and propose a PR to fix it to unblock this.

@vorburger vorburger self-assigned this Dec 28, 2021
@vorburger vorburger changed the title Fineract CD broken since ~2021-09-21 Fineract.dev Demo Server Continous Deployment (CD) is broken since ~2021-09-21 Dec 28, 2021
@vorburger
Copy link
Owner Author

I'll raise a new JIRA about it and propose a PR to fix it to unblock this.

https://issues.apache.org/jira/browse/FINERACT-1454

apache/fineract#2004

I'll pick this up when that is reviewed and merged.

@vorburger
Copy link
Owner Author

apache/fineract#2004 has been merged, but there is an unrelated other problem now:

Caused by: java.lang.RuntimeException: Failed to load driver class com.mysql.jdbc.Driver in either of HikariConfig class loader or Thread context classloader
	at com.zaxxer.hikari.HikariConfig.setDriverClassName(HikariConfig.java:491)
	at org.apache.fineract.infrastructure.core.config.HikariCpConfig.hikariConfig(HikariCpConfig.java:47)
	at org.apache.fineract.infrastructure.core.config.HikariCpConfig$$EnhancerBySpringCGLIB$$52f59a40.CGLIB$hikariConfig$1(<generated>)
	at org.apache.fineract.infrastructure.core.config.HikariCpConfig$$EnhancerBySpringCGLIB$$52f59a40$$FastClassBySpringCGLIB$$acb2f8e2.invoke(<generated>)
	at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244)
	at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331)
	at org.apache.fineract.infrastructure.core.config.HikariCpConfig$$EnhancerBySpringCGLIB$$52f59a40.hikariConfig(<generated>)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154)
	... 152 common frames omitted

on Fineract.dev I use fineract_tenants_driver: com.mysql.jdbc.Driver & DRIVERCLASS_NAME: com.mysql.jdbc.Driver.

Did Fineract recently change the JDBC driver that is shipped as part of the container image? I'll look into it.

@vorburger
Copy link
Owner Author

Did Fineract recently change the JDBC driver that is shipped as part of the container image? I'll look into it.

https://issues.apache.org/jira/browse/FINERACT-1456

I'll change the JDBC configuration used on https://www.fineract.dev.

@vorburger
Copy link
Owner Author

Phew, OK now there is ANOTHER (completely different problem) - see https://issues.apache.org/jira/browse/FINERACT-1457.

Caused by: org.flywaydb.core.internal.license.FlywayEditionUpgradeRequiredException: Flyway Teams Edition or MySQL upgrade required: MySQL 5.7 is no longer supported by Flyway Community Edition, but still supported by Flyway Teams Edition.

I'll attempt to upgrade the Google Cloud SQL MySQL DB version from 5.7 to 8.0.

@vorburger
Copy link
Owner Author

I'll attempt to upgrade the Google Cloud SQL MySQL DB version from 5.7 to 8.0.

That worked - and now I'm back to that Subquery returns more than 1 row from sql/migrations/core_db/V370__message_gateway_hook_template.sql, seen originally (above). I've filed https://issues.apache.org/jira/browse/FINERACT-1459 for that.

I'll work around that by reseting the DB.

@vorburger
Copy link
Owner Author

I'll work around that by reseting the DB.

That worked. https://demo.fineract.dev/fineract-provider/actuator/info now shows we're running the latest code again.

I'll now merge apache/fineract#2001, just to convince myself the automation still fully works.

@vorburger
Copy link
Owner Author

I'll now merge apache/fineract#2001, just to convince myself the automation still fully works.

That worked. https://demo.fineract.dev/fineract-provider/actuator/info now shows we're running code including apache/fineract@5efbd04.

Problem solved & case closed. PS: This was surprinsgly painful! 😒

@jdailey
Copy link

jdailey commented Dec 30, 2021

Thank you Michael for all of this digging. Any suggests for the future?

@vorburger
Copy link
Owner Author

Thank you Michael for all of this digging. Any suggests for the future?

I thought about that as well. I guess In a sort of "perfect world", any PR would be tested against Fineract.dev production upgrade. But this would require both time to set up and Cloud resources (you would need additional databases, because you would not want an in-Dev PR migration scripts to run on the "live" one, before merge) and some sort of queing infrastructure etc. etc. It's quite a bit of more work, more than I have time for. But even if I had the time and resources, I would have to argue that it would not be worth it, because: While it (this) was a onetime PITA, the CD to Fineract.dev has now (mostly) been "just working" almost completely "unattended" for 2 years already - in the overall bigger scheme of things, if we step back and squient at this, this is a actually huge success story! Unless it breaks with every other PR, I think a breakage once a year after major changes is "good enough".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants