springboot集成sharding-shardingsphere做分库分表(二)

目录

一、背景

二、撸码

1、建表

2、user表

(1)user表分库规则:

(2)user表分表规则:

3、order表

(1)order表分库规则:

(2)order表分表规则:

4、做sharding配置

5、做测试

6、做总结:

7、源码地址:

8、参考:


一、背景

springboot集成sharding-shardingsphere做分库分表(一)

上篇文章我们在springboot里集成了sharding-shardingsphere做分库分表,且测试完成了,当时我们每个库里的表名都是一样的,类似下面这样:

fincenter00库里有account_profile_00 -> account_profile_09;financial_order_00 -> financial_order_09

fincenter01库里有account_profile_00 -> account_profile_09;financial_order_00 -> financial_order_09

fincenter02库里有account_profile_00 -> account_profile_09;financial_order_00 -> financial_order_09

...............................

fincenter09库里有account_profile_00 -> account_profile_09;financial_order_00 -> financial_order_09

但是实际上公司里可能不是按照这个命名来的,像我们公司就是这样子的:

fincenter00库里有account_profile_00 -> account_profile_09;financial_order_00 -> financial_order_09

fincenter01库里有account_profile_10 -> account_profile_19;financial_order_10 -> financial_order_19

fincenter02库里有account_profile_20 -> account_profile_29;financial_order_20 -> financial_order_29

...............................

fincenter09库里有account_profile_90 -> account_profile_99;financial_order_90 -> financial_order_99

而且我们分库分表的话,是取id的倒数2~3位的,比如:userId=7977608474295009280,那么倒数2~3位为:28,会路由到 02库 28表 里。

那我们这篇文章就来以这种形式来集成。

注:我们这里的user_id和order_id都是自己用雪花算法生成的, 不再使用sharding-jdbc集成的雪花算法,那么我们的user_id和order_id就可以给String类型了,不需要再使用 int/long。

 

二、撸码

1、建表

建表语句见 gitee

2、user表

因为这里的user表的分表规则,不再是单纯的%2了,所以我们根据文档来看https://shardingsphere.apache.org/document/legacy/4.x/document/cn/features/sharding/concept/sharding/,单字段分片需要实现PreciseShardingAlgorithm接口

(1)user表分库规则:

package com.maple.sharding.config.sharding.user;

import com.alibaba.fastjson.JSONObject;
import org.apache.commons.lang3.StringUtils;
import org.apache.shardingsphere.api.sharding.standard.PreciseShardingAlgorithm;
import org.apache.shardingsphere.api.sharding.standard.PreciseShardingValue;
import org.springframework.stereotype.Component;

import java.util.Collection;

@Component
public class DatabasePreciseShardingAlgorithm implements PreciseShardingAlgorithm {

    @Override
    public String doSharding(Collection collection, PreciseShardingValue preciseShardingValue) {
        System.out.println("collection:" + JSONObject.toJSONString(collection) + ",shardingValues:" + JSONObject.toJSONString(preciseShardingValue));
        Collection<String> tables = collection;
        String shardingValue = getShardingValue(preciseShardingValue);
        return tables.stream().filter((table) -> table.endsWith(shardingValue)).findFirst().get();
    }

    /**
     * 获取倒数第3位
     *
     * @return
     */
    private String getShardingValue(PreciseShardingValue preciseShardingValue) {
        String tempShardingValue = String.valueOf(preciseShardingValue.getValue());
        return "0" + StringUtils.substring(tempShardingValue, tempShardingValue.length() - 3, tempShardingValue.length() - 2);
    }
}

(2)user表分表规则:

package com.maple.sharding.config.sharding.user;

import com.alibaba.fastjson.JSONObject;
import org.apache.commons.lang3.StringUtils;
import org.apache.shardingsphere.api.sharding.standard.PreciseShardingAlgorithm;
import org.apache.shardingsphere.api.sharding.standard.PreciseShardingValue;
import org.springframework.stereotype.Component;

import java.util.Collection;

@Component
public class TablePreciseShardingAlgorithm implements PreciseShardingAlgorithm {

    @Override
    public String doSharding(Collection collection, PreciseShardingValue preciseShardingValue) {
        System.out.println("collection:" + JSONObject.toJSONString(collection) + ",shardingValues:" + JSONObject.toJSONString(preciseShardingValue));
        Collection<String> tables = collection;
        String shardingValue = getShardingValue(preciseShardingValue);
        return tables.stream().filter((table) -> table.endsWith(shardingValue)).findFirst().get();
    }

    /**
     * 获取倒数第2~3位
     *
     * @return
     */
    private String getShardingValue(PreciseShardingValue preciseShardingValue) {
        String tempShardingValue = String.valueOf(preciseShardingValue.getValue());
        return StringUtils.substring(tempShardingValue, tempShardingValue.length() - 3, tempShardingValue.length() - 1);
    }
}

3、order表

order表的规则也不再是 % 2 或 % 10了,根据文档来看https://shardingsphere.apache.org/document/legacy/4.x/document/cn/features/sharding/concept/sharding/,多字段分片(order_id,user_id)需要实现ComplexKeysShardingAlgorithm接口

(1)order表分库规则:

package com.maple.sharding.config.sharding.order;

import com.alibaba.fastjson.JSONObject;
import org.apache.commons.lang3.StringUtils;
import org.apache.shardingsphere.api.sharding.complex.ComplexKeysShardingAlgorithm;
import org.apache.shardingsphere.api.sharding.complex.ComplexKeysShardingValue;
import org.springframework.stereotype.Component;

import java.util.ArrayList;
import java.util.Collection;
import java.util.LinkedList;
import java.util.Map;

@Component
public class DatabaseComplexKeysShardingAlgorithm implements ComplexKeysShardingAlgorithm {

    @Override
    public Collection<String> doSharding(Collection collection, ComplexKeysShardingValue shardingValues) {

        System.out.println("collection:" + JSONObject.toJSONString(collection) + ",shardingValues:" + JSONObject.toJSONString(shardingValues));

        Collection<String> routeTables = new ArrayList<>();

        Collection<String> tables = collection;

        String shardingValue = getShardingValue(shardingValues);

        routeTables.add(tables.stream().filter((table) -> table.endsWith(shardingValue)).findFirst().get());

        return routeTables;
    }

    /**
     * 获取倒数第3位
     *
     * @return
     */
    private String getShardingValue(ComplexKeysShardingValue shardingValues) {

        Map<String, LinkedList<String>> columnNameAndShardingValuesMap = shardingValues.getColumnNameAndShardingValuesMap();

        for (Map.Entry<String, LinkedList<String>> entry : columnNameAndShardingValuesMap.entrySet()) {

            //20210908745
            String tempShardingValue = entry.getValue().get(0);

            //7
            String shardingValue = StringUtils.substring(tempShardingValue, tempShardingValue.length() - 3, tempShardingValue.length() - 2);

            //07 库
            return "0" + shardingValue;
        }

        return null;
    }
}

(2)order表分表规则:

package com.maple.sharding.config.sharding.order;

import com.alibaba.fastjson.JSONObject;
import org.apache.commons.lang3.StringUtils;
import org.apache.shardingsphere.api.sharding.complex.ComplexKeysShardingAlgorithm;
import org.apache.shardingsphere.api.sharding.complex.ComplexKeysShardingValue;
import org.springframework.stereotype.Component;

import java.util.ArrayList;
import java.util.Collection;
import java.util.LinkedList;
import java.util.Map;

@Component
public class TableComplexKeysShardingAlgorithm implements ComplexKeysShardingAlgorithm {

    @Override
    public Collection<String> doSharding(Collection collection, ComplexKeysShardingValue shardingValues) {

        System.out.println("collection:" + JSONObject.toJSONString(collection) + ",shardingValues:" + JSONObject.toJSONString(shardingValues));

        Collection<String> routeTables = new ArrayList<>();

        Collection<String> tables = collection;

        String shardingValue = getShardingValue(shardingValues);

        routeTables.add(tables.stream().filter((table) -> table.endsWith(shardingValue)).findFirst().get());

        return routeTables;
    }

    /**
     * 获取倒数2~3位
     *
     * @return
     */
    private String getShardingValue(ComplexKeysShardingValue shardingValues) {

        Map<String, LinkedList<String>> columnNameAndShardingValuesMap = shardingValues.getColumnNameAndShardingValuesMap();

        for (Map.Entry<String, LinkedList<String>> entry : columnNameAndShardingValuesMap.entrySet()) {

            //20210908745
            String tempShardingValue = entry.getValue().get(0);

            //74
            String shardingValue = StringUtils.substring(tempShardingValue, tempShardingValue.length() - 3, tempShardingValue.length() - 1);

            //74 表
            return shardingValue;
        }
        return null;
    }
}

4、做sharding配置

spring.shardingsphere.sharding.default-data-source-name=fincenter00

spring.shardingsphere.datasource.names=fincenter00,fincenter01,fincenter02,fincenter03,fincenter04,fincenter05,fincenter06,fincenter07,fincenter08,fincenter09

#-----------------------------------------datasource start---------------------------------------------
#fincenter00
spring.shardingsphere.datasource.fincenter00.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.fincenter00.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.fincenter00.url=jdbc:mysql://localhost:3306/fincenter00?useSSL=false&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=UTC
spring.shardingsphere.datasource.fincenter00.username=root
spring.shardingsphere.datasource.fincenter00.password=123456

#fincenter01
spring.shardingsphere.datasource.fincenter01.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.fincenter01.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.fincenter01.url=jdbc:mysql://localhost:3306/fincenter01?useSSL=false&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=UTC
spring.shardingsphere.datasource.fincenter01.username=root
spring.shardingsphere.datasource.fincenter01.password=123456

#fincenter02
spring.shardingsphere.datasource.fincenter02.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.fincenter02.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.fincenter02.url=jdbc:mysql://localhost:3306/fincenter02?useSSL=false&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=UTC
spring.shardingsphere.datasource.fincenter02.username=root
spring.shardingsphere.datasource.fincenter02.password=123456

#fincenter03
spring.shardingsphere.datasource.fincenter03.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.fincenter03.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.fincenter03.url=jdbc:mysql://localhost:3306/fincenter03?useSSL=false&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=UTC
spring.shardingsphere.datasource.fincenter03.username=root
spring.shardingsphere.datasource.fincenter03.password=123456

#fincenter04
spring.shardingsphere.datasource.fincenter04.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.fincenter04.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.fincenter04.url=jdbc:mysql://localhost:3306/fincenter04?useSSL=false&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=UTC
spring.shardingsphere.datasource.fincenter04.username=root
spring.shardingsphere.datasource.fincenter04.password=123456

#fincenter05
spring.shardingsphere.datasource.fincenter05.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.fincenter05.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.fincenter05.url=jdbc:mysql://localhost:3306/fincenter05?useSSL=false&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=UTC
spring.shardingsphere.datasource.fincenter05.username=root
spring.shardingsphere.datasource.fincenter05.password=123456

#fincenter06
spring.shardingsphere.datasource.fincenter06.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.fincenter06.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.fincenter06.url=jdbc:mysql://localhost:3306/fincenter06?useSSL=false&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=UTC
spring.shardingsphere.datasource.fincenter06.username=root
spring.shardingsphere.datasource.fincenter06.password=123456

#fincenter07
spring.shardingsphere.datasource.fincenter07.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.fincenter07.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.fincenter07.url=jdbc:mysql://localhost:3306/fincenter07?useSSL=false&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=UTC
spring.shardingsphere.datasource.fincenter07.username=root
spring.shardingsphere.datasource.fincenter07.password=123456

#fincenter08
spring.shardingsphere.datasource.fincenter08.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.fincenter08.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.fincenter08.url=jdbc:mysql://localhost:3306/fincenter08?useSSL=false&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=UTC
spring.shardingsphere.datasource.fincenter08.username=root
spring.shardingsphere.datasource.fincenter08.password=123456

#fincenter09
spring.shardingsphere.datasource.fincenter09.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.fincenter09.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.fincenter09.url=jdbc:mysql://localhost:3306/fincenter09?useSSL=false&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=UTC
spring.shardingsphere.datasource.fincenter09.username=root
spring.shardingsphere.datasource.fincenter09.password=123456
#-----------------------------------------datasource end---------------------------------------------

#需要做sharding的表
spring.shardingsphere.sharding.tables.account_profile.actual-data-nodes=fincenter00.account_profile_0$->{0..9},fincenter01.account_profile_1$->{0..9},fincenter02.account_profile_2$->{0..9},fincenter03.account_profile_3$->{0..9},fincenter04.account_profile_4$->{0..9},fincenter02.account_profile_2$->{0..9},fincenter06.account_profile_6$->{0..9},fincenter07.account_profile_7$->{0..9},fincenter08.account_profile_8$->{0..9},fincenter09.account_profile_9$->{0..9}
spring.shardingsphere.sharding.tables.financial_order.actual-data-nodes=fincenter00.financial_order_0$->{0..9},fincenter01.financial_order_1$->{0..9},fincenter02.financial_order_2$->{0..9},fincenter03.financial_order_3$->{0..9},fincenter04.financial_order_4$->{0..9},fincenter05.financial_order_5$->{0..9},fincenter06.financial_order_6$->{0..9},fincenter07.financial_order_7$->{0..9},fincenter08.financial_order_8$->{0..9},fincenter09.financial_order_9$->{0..9}

#account_profile
spring.shardingsphere.sharding.tables.account_profile.database-strategy.standard.sharding-column=user_id
spring.shardingsphere.sharding.tables.account_profile.database-strategy.standard.precise-algorithm-class-name=com.maple.sharding.config.sharding.user.DatabasePreciseShardingAlgorithm

spring.shardingsphere.sharding.tables.account_profile.table-strategy.standard.sharding-column=user_id
spring.shardingsphere.sharding.tables.account_profile.table-strategy.standard.precise-algorithm-class-name=com.maple.sharding.config.sharding.user.TablePreciseShardingAlgorithm

#financial_order
spring.shardingsphere.sharding.tables.financial_order.database-strategy.complex.sharding-columns=user_id,order_id
spring.shardingsphere.sharding.tables.financial_order.database-strategy.complex.algorithm-class-name=com.maple.sharding.config.sharding.order.DatabaseComplexKeysShardingAlgorithm

spring.shardingsphere.sharding.tables.financial_order.table-strategy.complex.sharding-columns=user_id,order_id
spring.shardingsphere.sharding.tables.financial_order.table-strategy.complex.algorithm-class-name=com.maple.sharding.config.sharding.order.TableComplexKeysShardingAlgorithm


#开启sql 输出日志
spring.shardingsphere.props.sql.show=true
# 由于一个实体类对应两张表,所以会产生覆盖操作,加上这个配置解决覆盖问题
spring.main.allow-bean-definition-overriding=true


#mybatis
mybatis.mapper-locations=classpath:/mapper/*Mapper.xml
mybatis.type-aliases-package=com.maple.sharding.entity

5、做测试

6、做总结:

本篇文章在于实现不常规的、自定义的分片规则,可以根据sharding-jdbc里提供的接口来实现功能。

7、源码地址:

https://gitee.com/gane_maple/sharding

8、参考:

https://shardingsphere.apache.org/document/legacy/4.x/document/cn/features/sharding/concept/sharding/

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

maple 枫

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值