高性能系统设计
高性能系统设计是构建快速响应、高吞吐量系统的核心技术。通过合理的架构设计、算法优化、缓存策略和资源管理,可以构建出高性能的系统。
高性能 = 算法优化 + 缓存策略 + 并发处理 + 资源管理 + 监控调优
- 🧮 算法优化:选择合适的算法和数据结构,降低时间复杂度
- 💾 缓存策略:减少重复计算和I/O操作,提高数据访问速度
- 🔄 并发处理:充分利用多核CPU,提高系统吞吐量
- 📊 资源管理:合理分配和使用CPU、内存、I/O等资源
- 📈 监控调优:持续监控系统性能,及时发现和解决问题
1. 性能优化基础
1.1 性能指标
高性能系统的核心指标:
| 指标 | 说明 | 优化目标 | 衡量方式 |
|---|---|---|---|
| 响应时间 (Response Time) | 请求处理时间 | 降低延迟 | 毫秒/秒级 |
| 吞吐量 (Throughput) | 单位时间处理请求数 | 提高QPS | 请求/秒 |
| 并发数 (Concurrency) | 同时处理的请求数 | 提高并发能力 | 用户数/连接数 |
| 资源利用率 (Resource Utilization) | CPU、内存、网络使用率 | 提高效率 | 百分比 |
| 延迟 (Latency) | 请求在系统内传输时间 | 减少延迟 | 毫秒级 |
不同场景下的性能优化重点
-
Web应用:
- 响应时间:一般控制在200ms以内
- 吞吐量:根据业务需求,中小型应用通常1000-5000 QPS
- 并发数:中小型应用通常几百至几千并发连接
-
API服务:
- 响应时间:一般控制在50ms以内
- 吞吐量:通常要求更高,可达数万QPS
- 低延迟:对时间敏感的API要求更低的延迟
-
数据分析系统:
- 资源利用率:CPU和内存使用效率更为重要
- 批处理能力:单批次处理数据量和速度
- 可扩展性:随数据增长扩展能力
-
实时处理系统:
- 延迟:毫秒甚至微秒级的延迟要求
- 稳定性:避免延迟峰值和抖动
- 吞吐量:在保证低延迟前提下的处理能力
- 性能监控
- 指标收集
1@Component2public class PerformanceMonitor {3 4 private final MeterRegistry meterRegistry;5 6 public PerformanceMonitor(MeterRegistry meterRegistry) {7 this.meterRegistry = meterRegistry;8 }9 10 @Around("@annotation(org.springframework.web.bind.annotation.RequestMapping)")11 public Object monitorPerformance(ProceedingJoinPoint joinPoint) throws Throwable {12 Timer.Sample sample = Timer.start(meterRegistry);13 String methodName = joinPoint.getSignature().getName();14 15 try {16 Object result = joinPoint.proceed();17 18 // 记录成功请求19 sample.stop(Timer.builder("http.server.requests")20 .tag("method", methodName)21 .tag("status", "success")22 .register(meterRegistry));23 24 return result;25 } catch (Exception e) {26 // 记录失败请求27 sample.stop(Timer.builder("http.server.requests")28 .tag("method", methodName)29 .tag("status", "error")30 .register(meterRegistry));31 32 Counter.builder("error.count")33 .tag("method", methodName)34 .tag("exception", e.getClass().getSimpleName())35 .register(meterRegistry)36 .increment();37 38 throw e;39 }40 }41}1@RestController2@RequestMapping("/metrics")3public class MetricsController {4 5 private final MeterRegistry meterRegistry;6 7 public MetricsController(MeterRegistry meterRegistry) {8 this.meterRegistry = meterRegistry;9 10 // 注册JVM指标11 new JvmMemoryMetrics().bindTo(meterRegistry);12 new JvmGcMetrics().bindTo(meterRegistry);13 new JvmThreadMetrics().bindTo(meterRegistry);14 new ClassLoaderMetrics().bindTo(meterRegistry);15 16 // 注册系统指标17 new ProcessorMetrics().bindTo(meterRegistry);18 new UptimeMetrics().bindTo(meterRegistry);19 }20 21 @GetMapping("/custom")22 public Map<String, Double> getCustomMetrics() {23 Map<String, Double> metrics = new HashMap<>();24 25 // 提取关键指标26 metrics.put("response.time.avg", getTimerMetric("response.time", "avg"));27 metrics.put("response.time.max", getTimerMetric("response.time", "max"));28 metrics.put("throughput", getCounterRate("http.server.requests"));29 metrics.put("error.rate", calculateErrorRate());30 31 return metrics;32 }33 34 private double getTimerMetric(String name, String type) {35 Timer timer = meterRegistry.find(name).timer();36 if (timer == null) return 0.0;37 38 return "avg".equals(type) ? timer.mean(TimeUnit.MILLISECONDS) : timer.max(TimeUnit.MILLISECONDS);39 }40 41 private double getCounterRate(String name) {42 return meterRegistry.find(name).counter().count() / getUptime();43 }44 45 private double calculateErrorRate() {46 double total = meterRegistry.find("http.server.requests").counters().stream()47 .mapToDouble(Counter::count).sum();48 49 double errors = meterRegistry.find("http.server.requests")50 .tag("status", "error").counter().count();51 52 return total == 0 ? 0 : (errors / total) * 100;53 }54 55 private double getUptime() {56 return ManagementFactory.getRuntimeMXBean().getUptime() / 1000.0;57 }58}1.2 性能瓶颈分析
性能瓶颈分析是发现系统性能短板的重要步骤,可以通过监控工具和性能分析确定系统中的瓶颈点。
性能瓶颈识别方法
| 瓶颈类型 | 症状 | 常见原因 | 分析工具 |
|---|---|---|---|
| CPU瓶颈 | CPU使用率高,线程排队 | 计算密集型操作,死循环,线程争用 | JVisualVM,top,jstack |
| 内存瓶颈 | GC频繁,OOM异常 | 内存泄漏,大对象创建,Eden区过小 | JProfiler,jmap,MAT |
| 磁盘I/O瓶颈 | 磁盘读写频繁,响应慢 | 日志频繁写入,数据库索引不当 | iostat,iotop |
| 网络瓶颈 | 网络延迟高,丢包 | 带宽不足,网络拥塞,连接过多 | netstat,ping,wireshark |
| 代码瓶颈 | 热点方法执行慢 | 算法低效,锁竞争,资源泄漏 | JProfiler,YourKit |
- 性能分析器
- 热点分析
1@Component2public class PerformanceAnalyzer {3 4 private final ThreadMXBean threadMXBean = ManagementFactory.getThreadMXBean();5 private final MemoryMXBean memoryMXBean = ManagementFactory.getMemoryMXBean();6 7 public PerformanceReport analyzePerformance() {8 PerformanceReport report = new PerformanceReport();9 10 // CPU使用率分析11 report.setCpuUsage(analyzeCpuUsage());12 13 // 内存使用情况14 report.setMemoryUsage(analyzeMemoryUsage());15 16 // 线程状态分析17 report.setThreadInfo(analyzeThreadInfo());18 19 // GC情况分析20 report.setGcInfo(analyzeGcInfo());21 22 return report;23 }24 25 private double analyzeCpuUsage() {26 OperatingSystemMXBean osBean = ManagementFactory.getOperatingSystemMXBean();27 if (osBean instanceof com.sun.management.OperatingSystemMXBean) {28 return ((com.sun.management.OperatingSystemMXBean) osBean).getCpuLoad() * 100;29 }30 return 0.0;31 }32 33 private MemoryUsage analyzeMemoryUsage() {34 return memoryMXBean.getHeapMemoryUsage();35 }36 37 private ThreadInfo[] analyzeThreadInfo() {38 return threadMXBean.dumpAllThreads(false, false);39 }40 41 private List<GarbageCollectorMXBean> analyzeGcInfo() {42 return ManagementFactory.getGarbageCollectorMXBeans();43 }44}1@Component2public class HotspotAnalyzer {3 4 private final MeterRegistry meterRegistry;5 private final Map<String, Timer> methodTimers = new ConcurrentHashMap<>();6 7 public HotspotAnalyzer(MeterRegistry meterRegistry) {8 this.meterRegistry = meterRegistry;9 }10 11 @Around("execution(* com.example.service.*.*(..))")12 public Object analyzeMethodPerformance(ProceedingJoinPoint joinPoint) throws Throwable {13 String methodName = joinPoint.getSignature().toShortString();14 Timer timer = methodTimers.computeIfAbsent(methodName, 15 k -> Timer.builder("method.execution")16 .tag("method", methodName)17 .register(meterRegistry));18 19 return timer.recordCallable(() -> {20 try {21 return joinPoint.proceed();22 } catch (Throwable t) {23 if (t instanceof RuntimeException) {24 throw (RuntimeException) t;25 }26 throw new RuntimeException(t);27 }28 });29 }30 31 public List<MethodPerformance> getTopSlowMethods(int limit) {32 return methodTimers.entrySet().stream()33 .map(entry -> new MethodPerformance(34 entry.getKey(),35 entry.getValue().count(),36 entry.getValue().totalTime(TimeUnit.MILLISECONDS),37 entry.getValue().mean(TimeUnit.MILLISECONDS),38 entry.getValue().max(TimeUnit.MILLISECONDS)39 ))40 .sorted(Comparator.comparingDouble(MethodPerformance::getAvgTime).reversed())41 .limit(limit)42 .collect(Collectors.toList());43 }44 45 public static class MethodPerformance {46 private final String methodName;47 private final long invocationCount;48 private final double totalTime;49 private final double avgTime;50 private final double maxTime;51 52 // Constructor, getters53 // ...54 }55}- 生产环境影响:性能分析工具会对系统性能产生影响,在生产环境中使用需谨慎
- 采样频率:高频采样会提供更精确的数据,但也会产生更大的性能开销
- 全局视角:关注全局性能指标,避免只优化局部性能
- 系统状态:在不同负载条件下进行性能分析,特别是峰值负载时
- 持续监控:建立持续性能监控机制,及时发现性能退化
2. 算法优化
2.1 数据结构优化
选择合适的数据结构对系统性能有着至关重要的影响,特别是在高并发场景下。
- 高效数据结构
- 数据结构对比
1@Service2public class DataStructureOptimization {3 4 // 使用HashMap优化查找5 private Map<String, User> userCache = new ConcurrentHashMap<>();6 7 public User getUserById(String id) {8 return userCache.get(id);9 }10 11 // 使用TreeSet优化排序12 private TreeSet<Integer> sortedNumbers = new TreeSet<>();13 14 public void addNumber(int number) {15 sortedNumbers.add(number);16 }17 18 public int findClosestNumber(int target) {19 Integer ceiling = sortedNumbers.ceiling(target);20 Integer floor = sortedNumbers.floor(target);21 22 if (ceiling == null) return floor;23 if (floor == null) return ceiling;24 25 return Math.abs(ceiling - target) < Math.abs(floor - target) ? ceiling : floor;26 }27 28 // 使用LinkedHashMap实现LRU缓存29 private Map<String, String> lruCache = new LinkedHashMap<String, String>(100, 0.75f, true) {30 @Override31 protected boolean removeEldestEntry(Map.Entry<String, String> eldest) {32 return size() > 100;33 }34 };35 36 public String getValue(String key) {37 return lruCache.get(key);38 }39 40 public void putValue(String key, String value) {41 lruCache.put(key, value);42 }43}1@Service2public class DataStructureBenchmark {3 4 public void compareListPerformance() {5 // 准备测试数据6 int dataSize = 100000;7 Random random = new Random();8 9 // ArrayList性能测试10 List<Integer> arrayList = new ArrayList<>(dataSize);11 long startTime = System.nanoTime();12 13 for (int i = 0; i < dataSize; i++) {14 arrayList.add(random.nextInt());15 }16 17 long arrayListInsertTime = System.nanoTime() - startTime;18 19 // LinkedList性能测试20 List<Integer> linkedList = new LinkedList<>();21 startTime = System.nanoTime();22 23 for (int i = 0; i < dataSize; i++) {24 linkedList.add(random.nextInt());25 }26 27 long linkedListInsertTime = System.nanoTime() - startTime;28 29 // 随机访问性能测试30 startTime = System.nanoTime();31 for (int i = 0; i < 10000; i++) {32 arrayList.get(random.nextInt(dataSize));33 }34 long arrayListAccessTime = System.nanoTime() - startTime;35 36 startTime = System.nanoTime();37 for (int i = 0; i < 10000; i++) {38 linkedList.get(random.nextInt(dataSize));39 }40 long linkedListAccessTime = System.nanoTime() - startTime;41 42 // 输出结果43 System.out.println("ArrayList插入时间: " + arrayListInsertTime + "ns");44 System.out.println("LinkedList插入时间: " + linkedListInsertTime + "ns");45 System.out.println("ArrayList随机访问时间: " + arrayListAccessTime + "ns");46 System.out.println("LinkedList随机访问时间: " + linkedListAccessTime + "ns");47 }48}2.2 算法时间复杂度优化
优化算法时间复杂度是提高系统性能的关键手段,通过选择更高效的算法可以显著提升系统处理能力。
优化算法时间复杂度的常见策略:
- 减少嵌套循环:将O(n²)优化为O(n)或O(n log n)
- 使用哈希表:将查找复杂度从O(n)降至O(1)
- 使用二分查找:将有序数组查找从O(n)降至O(log n)
- 动态规划:避免重复计算,优化递归算法
- 贪心算法:在适当场景用局部最优解替代全局最优解
- 空间换时间:利用额外空间降低时间复杂度
- 优化前
- 优化后
- 性能对比
1// O(n²)复杂度 - 查找两数之和2public int[] findTwoSum(int[] nums, int target) {3 for (int i = 0; i < nums.length; i++) {4 for (int j = i + 1; j < nums.length; j++) {5 if (nums[i] + nums[j] == target) {6 return new int[] {i, j};7 }8 }9 }10 return null;11}1// O(n)复杂度 - 使用哈希表优化查找两数之和2public int[] findTwoSumOptimized(int[] nums, int target) {3 Map<Integer, Integer> numMap = new HashMap<>();4 for (int i = 0; i < nums.length; i++) {5 int complement = target - nums[i];6 if (numMap.containsKey(complement)) {7 return new int[] {numMap.get(complement), i};8 }9 numMap.put(nums[i], i);10 }11 return null;12}1@Service2public class AlgorithmBenchmark {3 4 public void compareTwoSumAlgorithms() {5 // 准备测试数据6 int[] nums = new int[10000];7 Random random = new Random();8 9 for (int i = 0; i < nums.length; i++) {10 nums[i] = random.nextInt(10000);11 }12 13 int target = nums[random.nextInt(nums.length)] + nums[random.nextInt(nums.length)];14 15 // 测试O(n²)算法16 long startTime = System.nanoTime();17 findTwoSum(nums, target);18 long bruteForceTime = System.nanoTime() - startTime;19 20 // 测试O(n)算法21 startTime = System.nanoTime();22 findTwoSumOptimized(nums, target);23 long optimizedTime = System.nanoTime() - startTime;24 25 // 输出结果26 System.out.println("O(n²)算法执行时间: " + bruteForceTime + "ns");27 System.out.println("O(n)算法执行时间: " + optimizedTime + "ns");28 System.out.println("性能提升: " + (bruteForceTime / optimizedTime) + "倍");29 }30}2.3 空间复杂度优化
在某些场景下,我们需要在空间复杂度和时间复杂度之间进行权衡,特别是在内存受限的环境中。
空间复杂度与时间复杂度的权衡
-
空间换时间:
- 预计算结果存储在内存中(如缓存、查找表)
- 使用额外数据结构加速处理(哈希表、树等)
- 适用于内存充足、性能要求高的场景
-
时间换空间:
- 使用迭代替代递归避免栈空间消耗
- 使用位操作节省空间
- 适用于内存受限场景
-
权衡因素:
- 硬件限制:内存大小、CPU性能
- 并发用户数:影响总内存需求
- 数据规模:处理数据量的大小
- 访问频率:高频访问适合空间换时间
1// 优化前:空间复杂度O(n)2public int fibonacci(int n) {3 int[] fib = new int[n + 1];4 fib[0] = 0;5 fib[1] = 1;6 for (int i = 2; i <= n; i++) {7 fib[i] = fib[i - 1] + fib[i - 2];8 }9 return fib[n];10}1112// 优化后:空间复杂度O(1)13public int fibonacciOptimized(int n) {14 if (n <= 1) return n;15 16 int prev = 0;17 int curr = 1;18 for (int i = 2; i <= n; i++) {19 int next = prev + curr;20 prev = curr;21 curr = next;22 }23 return curr;24}3. 缓存优化
缓存是提升系统性能最有效的手段之一,通过将频繁访问的数据存储在更快的介质中,可以显著减少访问延迟。
3.1 本地缓存
- Caffeine缓存
- Guava缓存
- EhCache
1@Configuration2public class CaffeineConfig {3 4 @Bean5 public Cache<String, Object> caffeineCache() {6 return Caffeine.newBuilder()7 .maximumSize(10_000)8 .expireAfterWrite(Duration.ofMinutes(5))9 .recordStats()10 .build();11 }12}1314@Service15public class CaffeineCacheService {16 17 @Autowired18 private Cache<String, Object> caffeineCache;19 20 @Autowired21 private DataRepository repository;22 23 public Object getData(String key) {24 // 从缓存获取25 Object value = caffeineCache.getIfPresent(key);26 if (value != null) {27 return value;28 }29 30 // 缓存未命中,从数据源获取31 value = repository.findByKey(key);32 if (value != null) {33 // 更新缓存34 caffeineCache.put(key, value);35 }36 37 return value;38 }39 40 public Map<String, Object> getCacheStats() {41 Map<String, Object> stats = new HashMap<>();42 stats.put("hitCount", caffeineCache.stats().hitCount());43 stats.put("missCount", caffeineCache.stats().missCount());44 stats.put("hitRate", caffeineCache.stats().hitRate());45 stats.put("evictionCount", caffeineCache.stats().evictionCount());46 stats.put("estimatedSize", caffeineCache.estimatedSize());47 return stats;48 }49}1@Configuration2public class GuavaCacheConfig {3 4 @Bean5 public LoadingCache<String, Object> guavaCache() {6 return CacheBuilder.newBuilder()7 .maximumSize(10_000)8 .expireAfterWrite(5, TimeUnit.MINUTES)9 .recordStats()10 .build(new CacheLoader<String, Object>() {11 @Override12 public Object load(String key) throws Exception {13 // 这里实现从数据源加载数据的逻辑14 return loadFromDataSource(key);15 }16 });17 }18 19 private Object loadFromDataSource(String key) {20 // 从数据源加载数据的实现21 return null;22 }23}2425@Service26public class GuavaCacheService {27 28 @Autowired29 private LoadingCache<String, Object> guavaCache;30 31 public Object getData(String key) {32 try {33 // 自动处理缓存未命中情况34 return guavaCache.get(key);35 } catch (ExecutionException e) {36 log.error("获取数据失败", e);37 return null;38 }39 }40 41 public Map<String, Object> getCacheStats() {42 CacheStats stats = guavaCache.stats();43 Map<String, Object> statsMap = new HashMap<>();44 statsMap.put("hitCount", stats.hitCount());45 statsMap.put("missCount", stats.missCount());46 statsMap.put("hitRate", stats.hitRate());47 statsMap.put("evictionCount", stats.evictionCount());48 statsMap.put("totalLoadTime", stats.totalLoadTime());49 statsMap.put("averageLoadPenalty", stats.averageLoadPenalty());50 return statsMap;51 }52}1@Configuration2@EnableCaching3public class EhCacheConfig {4 5 @Bean6 public CacheManager cacheManager() {7 CacheConfiguration<Object, Object> cacheConfiguration = CacheConfigurationBuilder8 .newCacheConfigurationBuilder(Object.class, Object.class,9 ResourcePoolsBuilder.heap(10000))10 .withExpiry(ExpiryPolicyBuilder.timeToLiveExpiration(Duration.ofMinutes(5)))11 .build();12 13 EhcacheCachingProvider provider = (EhcacheCachingProvider) Caching.getCachingProvider();14 org.ehcache.config.Configuration configuration = new DefaultConfiguration(provider.getDefaultClassLoader());15 16 EhcacheCacheManager ehCacheManager = new EhcacheCacheManager(CacheManagerBuilder17 .newCacheManagerBuilder()18 .withCache("default", cacheConfiguration)19 .build(true));20 21 return ehCacheManager;22 }23}2425@Service26public class EhCacheService {27 28 @Autowired29 private CacheManager cacheManager;30 31 @Autowired32 private DataRepository repository;33 34 @Cacheable(value = "default", key = "#key")35 public Object getData(String key) {36 // 缓存未命中时执行37 return repository.findByKey(key);38 }39 40 @CacheEvict(value = "default", key = "#key")41 public void invalidateCache(String key) {42 // 方法体可以为空,注解会处理缓存失效43 }44}- 内存压力:设置合理的最大大小,避免占用过多内存
- 过期策略:选择合适的过期策略(LRU、LFU、FIFO等)
- 缓存一致性:多实例场景下的缓存数据一致性问题
- 缓存穿透:对不存在的key进行重复查询导致缓存失效
- 缓存击穿:热点key过期瞬间导致大量请求直达数据库
- 缓存雪崩:大量缓存同时失效导致系统压力突增
3.2 分布式缓存
分布式缓存解决了本地缓存在多实例场景下的数据一致性问题,可以作为多个应用实例的共享缓存。
分布式缓存特点
| 特性 | 说明 |
|---|---|
| 数据一致性 | 多个应用实例访问相同的缓存数据,避免数据不一致 |
| 水平扩展 | 可以通过增加节点提升缓存容量和吞吐量 |
| 故障转移 | 支持节点故障时的自动故障转移,提高可用性 |
| 数据分区 | 通过哈希或一致性哈希等算法对数据进行分区存储 |
| 缓存协议 | 支持多种缓存访问协议,如Memcached、Redis协议等 |
- Redis缓存
- Redis集群
- Redis哨兵
1@Configuration2public class RedisConfig {3 4 @Bean5 public RedisConnectionFactory redisConnectionFactory() {6 LettuceConnectionFactory factory = new LettuceConnectionFactory();7 factory.setHostName("localhost");8 factory.setPort(6379);9 factory.afterPropertiesSet();10 return factory;11 }12 13 @Bean14 public RedisTemplate<String, Object> redisTemplate() {15 RedisTemplate<String, Object> template = new RedisTemplate<>();16 template.setConnectionFactory(redisConnectionFactory());17 template.setKeySerializer(new StringRedisSerializer());18 template.setValueSerializer(new Jackson2JsonRedisSerializer<>(Object.class));19 template.afterPropertiesSet();20 return template;21 }22 23 @Bean24 public RedisCacheManager cacheManager() {25 RedisCacheConfiguration cacheConfig = RedisCacheConfiguration.defaultCacheConfig()26 .entryTtl(Duration.ofMinutes(10))27 .serializeKeysWith(RedisSerializationContext.SerializationPair28 .fromSerializer(new StringRedisSerializer()))29 .serializeValuesWith(RedisSerializationContext.SerializationPair30 .fromSerializer(new GenericJackson2JsonRedisSerializer()));31 32 return RedisCacheManager.builder(redisConnectionFactory())33 .cacheDefaults(cacheConfig)34 .build();35 }36}3738@Service39public class RedisCacheService {40 41 @Autowired42 private RedisTemplate<String, Object> redisTemplate;43 44 @Autowired45 private DataRepository repository;46 47 public Object getData(String key) {48 // 从Redis获取数据49 Object value = redisTemplate.opsForValue().get(key);50 if (value != null) {51 return value;52 }53 54 // 缓存未命中,从数据库获取55 value = repository.findByKey(key);56 if (value != null) {57 // 更新缓存,设置过期时间58 redisTemplate.opsForValue().set(key, value, Duration.ofMinutes(10));59 }60 61 return value;62 }63 64 public void deleteData(String key) {65 // 删除数据库记录66 repository.deleteByKey(key);67 68 // 删除缓存69 redisTemplate.delete(key);70 }71}1@Configuration2public class RedisClusterConfig {3 4 @Bean5 public RedisConnectionFactory redisConnectionFactory() {6 RedisClusterConfiguration clusterConfig = new RedisClusterConfiguration();7 clusterConfig.addClusterNode(new RedisNode("redis-1", 6379));8 clusterConfig.addClusterNode(new RedisNode("redis-2", 6379));9 clusterConfig.addClusterNode(new RedisNode("redis-3", 6379));10 11 return new LettuceConnectionFactory(clusterConfig);12 }13 14 @Bean15 public RedisTemplate<String, Object> redisTemplate() {16 RedisTemplate<String, Object> template = new RedisTemplate<>();17 template.setConnectionFactory(redisConnectionFactory());18 template.setKeySerializer(new StringRedisSerializer());19 template.setValueSerializer(new GenericJackson2JsonRedisSerializer());20 template.afterPropertiesSet();21 return template;22 }23}1@Configuration2public class RedisSentinelConfig {3 4 @Bean5 public RedisConnectionFactory redisConnectionFactory() {6 RedisSentinelConfiguration sentinelConfig = new RedisSentinelConfiguration()7 .master("mymaster")8 .sentinel("redis-sentinel-1", 26379)9 .sentinel("redis-sentinel-2", 26379)10 .sentinel("redis-sentinel-3", 26379);11 12 return new LettuceConnectionFactory(sentinelConfig);13 }14 15 @Bean16 public RedisTemplate<String, Object> redisTemplate() {17 RedisTemplate<String, Object> template = new RedisTemplate<>();18 template.setConnectionFactory(redisConnectionFactory());19 template.setKeySerializer(new StringRedisSerializer());20 template.setValueSerializer(new GenericJackson2JsonRedisSerializer());21 template.afterPropertiesSet();22 return template;23 }24}3.3 缓存策略
缓存策略决定了缓存的使用方式和效果,选择合适的缓存策略对系统性能至关重要。
常见缓存策略
-
Cache-Aside(旁路缓存):
- 应用程序负责同时维护缓存和数据库
- 读取数据时先查缓存,缓存未命中再查数据库并更新缓存
- 写入数据时先更新数据库,然后更新或失效缓存
-
Read-Through:
- 缓存层负责从数据源加载数据
- 应用程序只与缓存层交互,无需关心数据源
- 缓存未命中时自动从数据源加载
-
Write-Through:
- 写入数据时先写入缓存,然后由缓存同步写入数据库
- 保证数据一致性,但写入性能较差
-
Write-Behind/Write-Back:
- 写入数据时只更新缓存,然后异步更新数据库
- 提高写入性能,但可能导致数据丢失
-
Write-Around:
- 写入数据时只更新数据库,不更新缓存
- 避免写入操作污染缓存,适用于读多写少场景
- Cache-Aside
- Write-Through
- Write-Behind
1@Service2public class CacheAsideService {3 4 @Autowired5 private RedisTemplate<String, Object> redisTemplate;6 7 @Autowired8 private UserRepository userRepository;9 10 // 读取操作11 public User getUser(Long id) {12 String key = "user:" + id;13 14 // 1. 从缓存读取15 User user = (User) redisTemplate.opsForValue().get(key);16 if (user != null) {17 return user;18 }19 20 // 2. 缓存未命中,从数据库读取21 user = userRepository.findById(id).orElse(null);22 if (user != null) {23 // 3. 更新缓存24 redisTemplate.opsForValue().set(key, user, Duration.ofMinutes(30));25 }26 27 return user;28 }29 30 // 写入操作31 @Transactional32 public void updateUser(User user) {33 // 1. 更新数据库34 userRepository.save(user);35 36 // 2. 删除缓存37 redisTemplate.delete("user:" + user.getId());38 }39}1@Service2public class WriteThroughService {3 4 @Autowired5 private RedisTemplate<String, Object> redisTemplate;6 7 @Autowired8 private UserRepository userRepository;9 10 // 写入操作11 @Transactional12 public void updateUser(User user) {13 // 1. 更新数据库14 userRepository.save(user);15 16 // 2. 更新缓存17 String key = "user:" + user.getId();18 redisTemplate.opsForValue().set(key, user, Duration.ofMinutes(30));19 }20}1@Service2public class WriteBehindService {3 4 @Autowired5 private RedisTemplate<String, Object> redisTemplate;6 7 @Autowired8 private UserRepository userRepository;9 10 @Autowired11 private KafkaTemplate<String, User> kafkaTemplate;12 13 // 写入操作14 public void updateUser(User user) {15 // 1. 更新缓存16 String key = "user:" + user.getId();17 redisTemplate.opsForValue().set(key, user, Duration.ofMinutes(30));18 19 // 2. 异步更新数据库20 kafkaTemplate.send("user-updates", user);21 }22 23 // 消费者处理数据库更新24 @KafkaListener(topics = "user-updates")25 public void processUserUpdate(User user) {26 userRepository.save(user);27 }28}4. 数据库优化
数据库往往是系统性能的瓶颈,优化数据库访问对提升整体性能至关重要。
4.1 索引优化
索引是提升数据库查询性能的关键技术,合理的索引设计可以大幅提升查询速度。
索引设计的核心原则:
- 高选择性优先:为选择性高的列创建索引(如主键、唯一约束列)
- 组合索引顺序:最左匹配原则,常用列放在前面
- 避免过度索引:索引也需要维护成本,过多索引会影响写性能
- 覆盖索引:使用包含所有查询所需字段的索引(索引覆盖)
- 索引列优化:避免对索引列进行函数操作,如
WHERE UPPER(name) = 'ABC' - 索引维护:定期分析和优化索引,删除未使用的索引
- 索引设计
- 索引最佳实践
- 索引分析
1-- 创建基本索引2CREATE INDEX idx_user_email ON users (email);34-- 创建组合索引5CREATE INDEX idx_user_name_city ON users (name, city);67-- 创建唯一索引8CREATE UNIQUE INDEX idx_user_username ON users (username);910-- 创建部分索引 (PostgreSQL)11CREATE INDEX idx_active_users ON users (created_at) WHERE active = true;1213-- 创建函数索引 (PostgreSQL)14CREATE INDEX idx_user_email_lower ON users (LOWER(email));1516-- 创建全文索引 (MySQL)17CREATE FULLTEXT INDEX idx_article_content ON articles (title, content);1@Repository2public interface UserRepository extends JpaRepository<User, Long> {3 4 // 使用索引的查询5 @Query("SELECT u FROM User u WHERE u.email = :email")6 Optional<User> findByEmail(String email);7 8 // 组合索引查询 - 利用idx_user_name_city9 @Query("SELECT u FROM User u WHERE u.name = :name AND u.city = :city")10 List<User> findByNameAndCity(String name, String city);11 12 // 范围查询 - 使用索引的最左前缀13 @Query("SELECT u FROM User u WHERE u.name = :name ORDER BY u.city")14 List<User> findByNameOrderByCity(String name);15 16 // 避免的查询模式 - 对索引列使用函数17 @Query("SELECT u FROM User u WHERE LOWER(u.email) = LOWER(:email)")18 Optional<User> findByEmailIgnoreCase(String email);19 20 // 更好的替代方案21 @Query(nativeQuery = true,22 value = "SELECT * FROM users WHERE email = :email COLLATE NOCASE")23 Optional<User> findByEmailCaseInsensitive(String email);24}1-- MySQL索引分析2EXPLAIN SELECT * FROM users WHERE email = 'user@example.com';34-- PostgreSQL索引分析5EXPLAIN ANALYZE SELECT * FROM users WHERE email = 'user@example.com';67-- 检查索引使用情况 (MySQL)8SELECT9 INDEX_NAME,10 TABLE_NAME,11 SEQ_IN_INDEX,12 COLUMN_NAME,13 CARDINALITY14FROM15 INFORMATION_SCHEMA.STATISTICS16WHERE17 TABLE_SCHEMA = 'your_database'18 AND TABLE_NAME = 'users'19ORDER BY20 INDEX_NAME,21 SEQ_IN_INDEX;22 23-- 检查未使用的索引 (MySQL)24SELECT25 objects.name AS table_name,26 indexes.name AS index_name,27 dm_db_index_usage_stats.user_seeks,28 dm_db_index_usage_stats.user_scans,29 dm_db_index_usage_stats.user_lookups30FROM31 sys.dm_db_index_usage_stats32 INNER JOIN sys.indexes ON dm_db_index_usage_stats.object_id = indexes.object_id33 AND dm_db_index_usage_stats.index_id = indexes.index_id34 INNER JOIN sys.objects ON indexes.object_id = objects.object_id35WHERE36 dm_db_index_usage_stats.user_seeks = 037 AND dm_db_index_usage_stats.user_scans = 038 AND dm_db_index_usage_stats.user_lookups = 039 AND objects.name = 'users';4.2 查询优化
优化SQL查询是提升数据库性能的重要手段,通过优化查询语句、避免常见陷阱,可以显著提升查询性能。
查询优化核心原则
- 只查询需要的列:避免
SELECT *,只查询需要的列 - 减少结果集大小:使用
LIMIT/TOP等限制结果集大小 - 优化JOIN:减少JOIN的表数量,选择正确的JOIN类型
- 分页优化:使用基于主键的分页,避免
OFFSET - 避免子查询:尽可能使用JOIN替代子查询
- 避免全表扫描:确保查询条件能够使用索引
- 适当反范式化:为了性能考虑,适当冗余存储部分数据
- 批量操作:使用批量插入/更新替代单条操作
- 查询优化示例
- ORM查询优化
- 批量操作
1-- 优化前:查询所有列,可能不必要2SELECT * FROM orders WHERE customer_id = 123;34-- 优化后:只查询需要的列5SELECT order_id, order_date, total_amount 6FROM orders 7WHERE customer_id = 123;89-- 优化前:子查询10SELECT * 11FROM orders 12WHERE customer_id IN (SELECT id FROM customers WHERE region = 'Europe');1314-- 优化后:使用JOIN15SELECT o.* 16FROM orders o 17JOIN customers c ON o.customer_id = c.id 18WHERE c.region = 'Europe';1920-- 优化前:OFFSET分页21SELECT * FROM products 22ORDER BY created_at 23LIMIT 20 OFFSET 1000;2425-- 优化后:基于ID分页26SELECT * FROM products 27WHERE id > 1000 28ORDER BY id 29LIMIT 20;3031-- 优化前:使用函数在WHERE子句32SELECT * FROM users 33WHERE YEAR(registration_date) = 2023;3435-- 优化后:避免在索引列上使用函数36SELECT * FROM users 37WHERE registration_date >= '2023-01-01' 38AND registration_date < '2024-01-01';1@Repository2public class OrderRepository {3 4 @PersistenceContext5 private EntityManager entityManager;6 7 // 优化前:获取所有字段8 public List<Order> findByCustomerId(Long customerId) {9 return entityManager.createQuery(10 "SELECT o FROM Order o WHERE o.customer.id = :customerId", Order.class)11 .setParameter("customerId", customerId)12 .getResultList();13 }14 15 // 优化后:仅查询需要的字段16 public List<OrderProjection> findOrderProjectionsByCustomerId(Long customerId) {17 return entityManager.createQuery(18 "SELECT new com.example.OrderProjection(o.id, o.orderDate, o.totalAmount) " +19 "FROM Order o WHERE o.customer.id = :customerId", OrderProjection.class)20 .setParameter("customerId", customerId)21 .getResultList();22 }23 24 // 优化前:使用JOIN FETCH加载所有关联25 public List<Order> findOrdersWithDetails(Long customerId) {26 return entityManager.createQuery(27 "SELECT o FROM Order o " +28 "JOIN FETCH o.items " +29 "JOIN FETCH o.customer " +30 "WHERE o.customer.id = :customerId", Order.class)31 .setParameter("customerId", customerId)32 .getResultList();33 }34 35 // 优化后:使用分页和懒加载36 public List<Order> findOrdersWithPagination(Long customerId, int page, int size) {37 return entityManager.createQuery(38 "SELECT o FROM Order o " +39 "WHERE o.customer.id = :customerId", Order.class)40 .setParameter("customerId", customerId)41 .setFirstResult(page * size)42 .setMaxResults(size)43 .getResultList();44 }45}1@Service2@Transactional3public class BatchOperationService {4 5 @PersistenceContext6 private EntityManager entityManager;7 8 // 批量插入9 public void batchInsert(List<User> users) {10 final int batchSize = 100;11 12 for (int i = 0; i < users.size(); i++) {13 entityManager.persist(users.get(i));14 15 if (i % batchSize == 0 && i > 0) {16 // 每插入batchSize条数据,刷新并清理EntityManager17 entityManager.flush();18 entityManager.clear();19 }20 }21 entityManager.flush();22 entityManager.clear();23 }24 25 // 批量更新26 public void batchUpdate(List<UserUpdateDTO> updates) {27 final int batchSize = 100;28 29 Query query = entityManager.createQuery(30 "UPDATE User u SET u.status = :status WHERE u.id = :id");31 32 for (int i = 0; i < updates.size(); i++) {33 UserUpdateDTO update = updates.get(i);34 query.setParameter("status", update.getStatus())35 .setParameter("id", update.getId())36 .executeUpdate();37 38 if (i % batchSize == 0 && i > 0) {39 entityManager.flush();40 entityManager.clear();41 }42 }43 entityManager.flush();44 entityManager.clear();45 }46 47 // JDBC批量插入48 public void jdbcBatchInsert(List<User> users) {49 JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource);50 51 jdbcTemplate.batchUpdate(52 "INSERT INTO users (name, email, status) VALUES (?, ?, ?)",53 new BatchPreparedStatementSetter() {54 @Override55 public void setValues(PreparedStatement ps, int i) throws SQLException {56 User user = users.get(i);57 ps.setString(1, user.getName());58 ps.setString(2, user.getEmail());59 ps.setString(3, user.getStatus());60 }61 62 @Override63 public int getBatchSize() {64 return users.size();65 }66 });67 }68}4.3 连接池优化
数据库连接池优化是提升数据库访问性能的重要手段,连接池避免了频繁创建和销毁数据库连接的开销。
- HikariCP配置
- DBCP2配置
- 连接池监控
1@Configuration2public class HikariConfig {3 4 @Bean5 public DataSource dataSource() {6 com.zaxxer.hikari.HikariConfig config = new com.zaxxer.hikari.HikariConfig();7 config.setJdbcUrl("jdbc:mysql://localhost:3306/mydb");8 config.setUsername("user");9 config.setPassword("password");10 config.setDriverClassName("com.mysql.cj.jdbc.Driver");11 12 // 连接池核心参数13 config.setMaximumPoolSize(20);14 config.setMinimumIdle(5);15 config.setIdleTimeout(30000);16 config.setConnectionTimeout(10000);17 config.setMaxLifetime(1800000);18 19 // 性能优化参数20 config.addDataSourceProperty("cachePrepStmts", "true");21 config.addDataSourceProperty("prepStmtCacheSize", "250");22 config.addDataSourceProperty("prepStmtCacheSqlLimit", "2048");23 config.addDataSourceProperty("useServerPrepStmts", "true");24 25 return new HikariDataSource(config);26 }27}1@Configuration2public class Dbcp2Config {3 4 @Bean5 public DataSource dataSource() {6 BasicDataSource dataSource = new BasicDataSource();7 dataSource.setDriverClassName("com.mysql.cj.jdbc.Driver");8 dataSource.setUrl("jdbc:mysql://localhost:3306/mydb");9 dataSource.setUsername("user");10 dataSource.setPassword("password");11 12 // 连接池大小13 dataSource.setInitialSize(5);14 dataSource.setMaxTotal(20);15 dataSource.setMaxIdle(10);16 dataSource.setMinIdle(5);17 18 // 连接有效性检查19 dataSource.setTestOnBorrow(true);20 dataSource.setValidationQuery("SELECT 1");21 dataSource.setValidationQueryTimeout(5);22 23 // 连接回收参数24 dataSource.setMaxWaitMillis(10000);25 dataSource.setRemoveAbandonedTimeout(60);26 dataSource.setRemoveAbandonedOnBorrow(true);27 dataSource.setRemoveAbandonedOnMaintenance(true);28 29 return dataSource;30 }31}1@Component2@ManagedResource(objectName = "com.example:type=ConnectionPool")3public class ConnectionPoolMonitor {4 5 private final HikariDataSource dataSource;6 7 public ConnectionPoolMonitor(DataSource dataSource) {8 this.dataSource = (HikariDataSource) dataSource;9 }10 11 @ManagedAttribute12 public int getTotalConnections() {13 return dataSource.getHikariPoolMXBean().getTotalConnections();14 }15 16 @ManagedAttribute17 public int getActiveConnections() {18 return dataSource.getHikariPoolMXBean().getActiveConnections();19 }20 21 @ManagedAttribute22 public int getIdleConnections() {23 return dataSource.getHikariPoolMXBean().getIdleConnections();24 }25 26 @ManagedAttribute27 public int getThreadsAwaitingConnection() {28 return dataSource.getHikariPoolMXBean().getThreadsAwaitingConnection();29 }30 31 @Scheduled(fixedRate = 60000)32 public void logPoolStats() {33 log.info("Connection Pool Stats - Total: {}, Active: {}, Idle: {}, Waiting: {}",34 getTotalConnections(), getActiveConnections(), getIdleConnections(), 35 getThreadsAwaitingConnection());36 }37}- 连接池大小:根据硬件资源和并发量合理设置,通常
连接数 = ((CPU核心数 * 2) + 有效磁盘数) - 最小空闲连接:保持一定数量的空闲连接,减少连接创建开销
- 连接超时:设置合理的连接获取超时时间,避免线程长时间等待
- 连接有效性检查:定期检查连接有效性,但不要过于频繁
- 预编译语句缓存:启用预编译语句缓存,提高查询性能
- 监控与告警:实时监控连接池状态,及时发现问题
5. 代码层面优化
5.1 异步编程
异步编程是提升系统吞吐量的重要手段,通过将阻塞操作异步化,可以提高系统资源利用率。
- CompletableFuture
- Spring WebFlux
- Java虚拟线程
1@Service2public class AsyncService {3 4 @Autowired5 private RestTemplate restTemplate;6 7 @Autowired8 private Executor executor;9 10 @Async11 public CompletableFuture<UserInfo> getUserInfo(Long userId) {12 return CompletableFuture.supplyAsync(() -> {13 // 模拟远程调用14 return restTemplate.getForObject(15 "https://api.example.com/users/{id}", UserInfo.class, userId);16 }, executor);17 }18 19 @Async20 public CompletableFuture<List<Order>> getUserOrders(Long userId) {21 return CompletableFuture.supplyAsync(() -> {22 // 模拟远程调用23 return restTemplate.getForObject(24 "https://api.example.com/users/{id}/orders", 25 new ParameterizedTypeReference<List<Order>>() {}, 26 userId);27 }, executor);28 }29 30 public CompletableFuture<UserDetails> getUserDetails(Long userId) {31 CompletableFuture<UserInfo> userInfoFuture = getUserInfo(userId);32 CompletableFuture<List<Order>> ordersFuture = getUserOrders(userId);33 34 return CompletableFuture.allOf(userInfoFuture, ordersFuture)35 .thenApply(v -> {36 UserInfo userInfo = userInfoFuture.join();37 List<Order> orders = ordersFuture.join();38 return new UserDetails(userInfo, orders);39 });40 }41}1@RestController2@RequestMapping("/api/users")3public class UserController {4 5 @Autowired6 private UserService userService;7 8 @GetMapping("/{id}")9 public Mono<UserDetails> getUserDetails(@PathVariable Long id) {10 Mono<UserInfo> userInfo = userService.getUserInfo(id);11 Mono<List<Order>> orders = userService.getUserOrders(id);12 13 return Mono.zip(userInfo, orders, UserDetails::new);14 }15}1617@Service18public class UserService {19 20 @Autowired21 private WebClient webClient;22 23 public Mono<UserInfo> getUserInfo(Long userId) {24 return webClient.get()25 .uri("https://api.example.com/users/{id}", userId)26 .retrieve()27 .bodyToMono(UserInfo.class);28 }29 30 public Mono<List<Order>> getUserOrders(Long userId) {31 return webClient.get()32 .uri("https://api.example.com/users/{id}/orders", userId)33 .retrieve()34 .bodyToFlux(Order.class)35 .collectList();36 }37}1@Service2public class VirtualThreadService {3 4 @Autowired5 private UserRepository userRepository;6 7 @Autowired8 private OrderRepository orderRepository;9 10 // 使用Java 19+的虚拟线程11 public UserDetails getUserDetails(Long userId) throws ExecutionException, InterruptedException {12 try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {13 Future<User> userFuture = executor.submit(() -> userRepository.findById(userId).orElse(null));14 Future<List<Order>> ordersFuture = executor.submit(() -> orderRepository.findByUserId(userId));15 16 User user = userFuture.get();17 List<Order> orders = ordersFuture.get();18 19 return new UserDetails(user, orders);20 }21 }22 23 // 处理大量并发请求24 public void processRequests(List<Long> userIds) {25 try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {26 List<Future<UserDetails>> futures = userIds.stream()27 .map(id -> executor.submit(() -> getUserDetailsBlocking(id)))28 .collect(Collectors.toList());29 30 for (Future<UserDetails> future : futures) {31 UserDetails details = future.get();32 processUserDetails(details);33 }34 }35 }36 37 private UserDetails getUserDetailsBlocking(Long userId) {38 User user = userRepository.findById(userId).orElse(null);39 List<Order> orders = orderRepository.findByUserId(userId);40 return new UserDetails(user, orders);41 }42 43 private void processUserDetails(UserDetails details) {44 // 处理用户详情45 }46}5.2 并行处理
并行处理是充分利用多核CPU资源的重要方式,通过将任务分解为多个子任务并行执行,可以提高处理速度。
- 并行流
- Fork/Join框架
- 批量异步处理
1@Service2public class ParallelProcessingService {3 4 // 并行处理集合5 public List<ProcessedData> processDataParallel(List<RawData> dataList) {6 return dataList.parallelStream()7 .map(this::processData)8 .collect(Collectors.toList());9 }10 11 // 并行计算统计信息12 public Map<String, Double> calculateStatistics(List<SalesRecord> records) {13 DoubleSummaryStatistics stats = records.parallelStream()14 .mapToDouble(SalesRecord::getAmount)15 .summaryStatistics();16 17 Map<String, Double> result = new HashMap<>();18 result.put("sum", stats.getSum());19 result.put("average", stats.getAverage());20 result.put("max", stats.getMax());21 result.put("min", stats.getMin());22 23 return result;24 }25 26 // 分组处理27 public Map<String, List<Product>> groupProductsByCategory(List<Product> products) {28 return products.parallelStream()29 .collect(Collectors.groupingByConcurrent(Product::getCategory));30 }31 32 private ProcessedData processData(RawData data) {33 // 处理单个数据项的逻辑34 return new ProcessedData(data);35 }36}1@Component2public class DataProcessingTask extends RecursiveTask<List<ProcessedData>> {3 4 private static final int THRESHOLD = 100;5 private final List<RawData> dataList;6 private final int start;7 private final int end;8 9 public DataProcessingTask(List<RawData> dataList, int start, int end) {10 this.dataList = dataList;11 this.start = start;12 this.end = end;13 }14 15 @Override16 protected List<ProcessedData> compute() {17 int length = end - start;18 19 if (length <= THRESHOLD) {20 // 小任务直接处理21 return processSequentially();22 }23 24 // 大任务分解为子任务25 int middle = start + length / 2;26 27 DataProcessingTask leftTask = new DataProcessingTask(dataList, start, middle);28 DataProcessingTask rightTask = new DataProcessingTask(dataList, middle, end);29 30 // 分叉执行子任务31 leftTask.fork();32 List<ProcessedData> rightResult = rightTask.compute();33 List<ProcessedData> leftResult = leftTask.join();34 35 // 合并结果36 List<ProcessedData> result = new ArrayList<>(leftResult);37 result.addAll(rightResult);38 return result;39 }40 41 private List<ProcessedData> processSequentially() {42 List<ProcessedData> result = new ArrayList<>();43 for (int i = start; i < end; i++) {44 result.add(processData(dataList.get(i)));45 }46 return result;47 }48 49 private ProcessedData processData(RawData data) {50 // 处理单个数据项的逻辑51 return new ProcessedData(data);52 }53}5455@Service56public class ForkJoinService {57 58 private final ForkJoinPool forkJoinPool = new ForkJoinPool();59 60 public List<ProcessedData> processDataParallel(List<RawData> dataList) {61 return forkJoinPool.invoke(new DataProcessingTask(dataList, 0, dataList.size()));62 }63}1@Service2public class BatchProcessingService {3 4 @Autowired5 private Executor executor;6 7 public List<ProcessedData> processDataInBatches(List<RawData> dataList, int batchSize) {8 // 分批处理9 List<List<RawData>> batches = splitIntoBatches(dataList, batchSize);10 11 // 创建每批数据的CompletableFuture12 List<CompletableFuture<List<ProcessedData>>> futures = batches.stream()13 .map(batch -> CompletableFuture.supplyAsync(() -> processBatch(batch), executor))14 .collect(Collectors.toList());15 16 // 等待所有批次完成并合并结果17 return CompletableFuture.allOf(futures.toArray(new CompletableFuture[0]))18 .thenApply(v -> futures.stream()19 .flatMap(future -> future.join().stream())20 .collect(Collectors.toList()))21 .join();22 }23 24 private List<ProcessedData> processBatch(List<RawData> batch) {25 return batch.stream()26 .map(this::processData)27 .collect(Collectors.toList());28 }29 30 private List<List<RawData>> splitIntoBatches(List<RawData> dataList, int batchSize) {31 List<List<RawData>> batches = new ArrayList<>();32 for (int i = 0; i < dataList.size(); i += batchSize) {33 int endIndex = Math.min(i + batchSize, dataList.size());34 batches.add(dataList.subList(i, endIndex));35 }36 return batches;37 }38 39 private ProcessedData processData(RawData data) {40 // 处理单个数据项的逻辑41 return new ProcessedData(data);42 }43}5.3 内存优化
内存优化是提高系统性能和稳定性的重要方面,通过合理管理内存使用,可以避免内存泄漏和OOM错误。
内存优化的核心原则:
- 对象重用:避免频繁创建和销毁对象,尤其是大对象
- 对象池化:使用对象池重用昂贵的对象,如连接、线程等
- 合理缓存:缓存常用数据,但要设置合理的缓存大小
- 避免内存泄漏:正确关闭资源,避免强引用持有不再使用的对象
- 内存分配优化:减少大对象分配,避免频繁GC
- 字符串优化:合理使用字符串,避免频繁字符串拼接
- 对象池化
- 字符串优化
- 内存泄漏避免
1@Configuration2public class ObjectPoolConfig {3 4 @Bean5 public GenericObjectPool<ExpensiveObject> expensiveObjectPool() {6 GenericObjectPoolConfig<ExpensiveObject> config = new GenericObjectPoolConfig<>();7 config.setMaxTotal(20);8 config.setMaxIdle(10);9 config.setMinIdle(5);10 config.setTestOnBorrow(true);11 12 return new GenericObjectPool<>(new ExpensiveObjectFactory(), config);13 }14}1516@Component17public class ExpensiveObjectFactory extends BasePooledObjectFactory<ExpensiveObject> {18 19 @Override20 public ExpensiveObject create() {21 return new ExpensiveObject();22 }23 24 @Override25 public PooledObject<ExpensiveObject> wrap(ExpensiveObject obj) {26 return new DefaultPooledObject<>(obj);27 }28 29 @Override30 public void passivateObject(PooledObject<ExpensiveObject> p) {31 p.getObject().reset();32 }33 34 @Override35 public boolean validateObject(PooledObject<ExpensiveObject> p) {36 return p.getObject().isValid();37 }38}3940@Service41public class ObjectPoolService {42 43 @Autowired44 private GenericObjectPool<ExpensiveObject> objectPool;45 46 public void processWithPooledObject() {47 ExpensiveObject obj = null;48 try {49 obj = objectPool.borrowObject();50 obj.doSomething();51 } catch (Exception e) {52 log.error("Error borrowing object from pool", e);53 } finally {54 if (obj != null) {55 objectPool.returnObject(obj);56 }57 }58 }59}1@Service2public class StringOptimizationService {3 4 // 优化前:使用+拼接字符串5 public String concatenateBad(List<String> strings) {6 String result = "";7 for (String s : strings) {8 result = result + s;9 }10 return result;11 }12 13 // 优化后:使用StringBuilder14 public String concatenateGood(List<String> strings) {15 StringBuilder sb = new StringBuilder(strings.size() * 16); // 预估大小16 for (String s : strings) {17 sb.append(s);18 }19 return sb.toString();20 }21 22 // 优化后:使用StringJoiner23 public String joinStrings(List<String> strings, String delimiter) {24 StringJoiner joiner = new StringJoiner(delimiter);25 for (String s : strings) {26 joiner.add(s);27 }28 return joiner.toString();29 }30 31 // 优化后:使用String.join32 public String joinWithApi(List<String> strings, String delimiter) {33 return String.join(delimiter, strings);34 }35 36 // 优化后:使用流式API37 public String joinWithStream(List<String> strings, String delimiter) {38 return strings.stream().collect(Collectors.joining(delimiter));39 }40}1@Service2public class MemoryLeakPreventionService {3 4 // 使用WeakHashMap防止内存泄漏5 private final Map<Object, Object> cache = Collections.synchronizedMap(new WeakHashMap<>());6 7 // 使用软引用缓存8 private final Map<String, SoftReference<byte[]>> dataCache = new ConcurrentHashMap<>();9 10 // 缓存数据,使用软引用11 public void cacheData(String key, byte[] data) {12 dataCache.put(key, new SoftReference<>(data));13 }14 15 // 获取缓存数据,处理引用被回收的情况16 public byte[] getData(String key) {17 SoftReference<byte[]> reference = dataCache.get(key);18 if (reference == null) {19 return null;20 }21 22 byte[] data = reference.get();23 if (data == null) {24 // 引用已被回收,从数据源重新加载25 dataCache.remove(key);26 return null;27 }28 29 return data;30 }31 32 // 使用try-with-resources自动关闭资源33 public String readFile(Path path) {34 try (BufferedReader reader = Files.newBufferedReader(path)) {35 return reader.lines().collect(Collectors.joining("\n"));36 } catch (IOException e) {37 log.error("Error reading file", e);38 return null;39 }40 }41 42 // 注意清理ThreadLocal变量43 private static final ThreadLocal<UserContext> userContext = ThreadLocal.withInitial(UserContext::new);44 45 public void processRequest() {46 try {47 userContext.get().setUser(getCurrentUser());48 // 处理请求49 } finally {50 // 清理ThreadLocal变量,防止内存泄漏51 userContext.remove();52 }53 }54}6. 面试题精选
Q: 如何提升系统的响应时间?
A: 提升系统响应时间的方法包括:
-
缓存优化:
- 使用多级缓存(本地缓存、分布式缓存)
- 预热缓存,避免冷启动
- 缓存热点数据,减少数据库访问
-
数据库优化:
- 优化索引设计,确保查询使用索引
- 优化SQL查询,避免全表扫描和复杂连接
- 读写分离,减轻主库负担
-
代码层面优化:
- 异步处理非实时操作
- 优化算法,降低时间复杂度
- 使用批处理代替单条处理
-
网络优化:
- 使用CDN加速静态资源
- 减少HTTP请求数量和大小
- 启用HTTP/2和压缩
-
JVM优化:
- 调整GC策略,减少停顿时间
- 适当增加堆内存,减少GC频率
- 使用JIT编译优化热点代码
Q: 如何设计一个高性能的缓存系统?
A: 设计高性能缓存系统的关键点:
-
多级缓存架构:
- L1:本地内存缓存(Caffeine、Guava)
- L2:分布式缓存(Redis、Memcached)
- L3:持久化存储(数据库)
-
缓存策略:
- 数据一致性策略:Cache-Aside、Read-Through、Write-Through等
- 过期策略:TTL、LRU、LFU等
- 预加载策略:系统启动时预热缓存
-
缓存穿透、击穿、雪崩防护:
- 布隆过滤器预防缓存穿透
- 互斥锁预防缓存击穿
- 随机过期时间预防缓存雪崩
-
监控与维护:
- 缓存命中率监控
- 缓存容量和内存使用监控
- 缓存自动扩容和收缩机制
-
分布式场景考虑:
- 一致性哈希分片
- 主从复制和故障转移
- 跨区域数据同步
Q: 什么是JVM性能调优?如何进行?
A: JVM性能调优是优化Java虚拟机运行参数以提高应用性能的过程:
-
内存分配调优:
- 设置合适的堆内存大小(-Xms、-Xmx)
- 调整新生代和老年代比例(-XX:NewRatio)
- 调整Eden区和Survivor区比例(-XX:SurvivorRatio)
-
垃圾回收调优:
- 选择合适的垃圾收集器(Serial、Parallel、CMS、G1)
- 调整GC触发阈值和频率
- 设置GC日志和监控
-
JIT编译优化:
- 启用方法内联(-XX:+Inline)
- 优化逃逸分析(-XX:+DoEscapeAnalysis)
- 调整代码缓存大小(-XX:ReservedCodeCacheSize)
-
调优步骤:
- 收集性能数据:JVM堆转储、GC日志、线程快照
- 分析问题:内存泄漏、GC频繁、线程阻塞等
- 调整参数:根据分析结果调整JVM参数
- 验证效果:性能测试对比调优前后差异
- 持续优化:根据实际运行情况持续调优
-
常用工具:
- JVisualVM:内存分析、线程分析
- JMC(Java Mission Control):实时监控
- MAT(Memory Analyzer Tool):堆转储分析
- GCViewer:GC日志分析
Q: 如何处理系统的性能瓶颈?
A: 处理系统性能瓶颈的步骤:
-
瓶颈识别:
- 监控系统各项指标(CPU、内存、磁盘I/O、网络I/O等)
- 分析慢查询日志和性能追踪数据
- 通过压力测试发现系统容量极限
-
CPU瓶颈:
- 优化算法,降低时间复杂度
- 使用并行处理提高CPU利用率
- 引入缓存减少重复计算
- 异步处理非实时任务
-
内存瓶颈:
- 检查并修复内存泄漏
- 优化对象创建和销毁,使用对象池
- 调整JVM堆内存配置
- 使用弱引用和软引用管理缓存
-
磁盘I/O瓶颈:
- 使用缓存减少磁盘读取
- 批量处理文件操作
- 使用异步I/O和内存映射文件
- 考虑使用SSD或分布式存储
-
网络I/O瓶颈:
- 减少网络往返次数
- 使用连接池复用连接
- 压缩传输数据
- 使用异步非阻塞I/O
-
数据库瓶颈:
- 优化索引和SQL查询
- 实施读写分离和分库分表
- 使用数据库连接池
- 引入缓存减少数据库访问
-
水平扩展:
- 增加更多服务器实例
- 实施负载均衡
- 使用分布式架构
- 引入服务网格和容器编排
- 全面监控:建立完善的监控系统,及时发现性能问题
- 算法优化:选择合适的算法和数据结构,降低时间复杂度
- 缓存策略:合理使用多级缓存,减少重复计算和I/O操作
- 并发处理:充分利用多核CPU,提高系统吞吐量
- 数据库优化:索引设计、查询优化、分库分表等策略
- 资源管理:连接池、线程池、对象池等资源复用机制
- 内存管理:避免内存泄漏,优化对象创建和销毁
- 异步处理:使用异步编程模型,提高系统响应性能
通过本章的学习,你应该已经掌握了高性能系统设计的核心概念和优化策略。高性能系统设计是一个持续优化的过程,需要不断监控、分析和改进。在实际项目中,应根据具体场景选择合适的优化方法,在性能、可用性、可维护性之间找到平衡点。
参与讨论