Browse Source

优化读取全部sheet方案

pull/724/head
Jiaju Zhuang 5 years ago
parent
commit
2320becd47
  1. 9
      README.md
  2. 5
      docs/API.md
  3. 116
      problem.md
  4. 1365
      quickstart.md
  5. 2
      src/main/java/com/alibaba/excel/analysis/ExcelAnalyserImpl.java
  6. 10
      src/main/java/com/alibaba/excel/read/metadata/holder/ReadWorkbookHolder.java
  7. 8
      src/main/java/com/alibaba/excel/support/ExcelTypeEnum.java
  8. 2
      src/main/java/com/alibaba/excel/write/metadata/WriteWorkbook.java
  9. 6
      src/test/java/com/alibaba/easyexcel/test/core/encrypt/EncryptDataTest.java
  10. 2
      src/test/java/com/alibaba/easyexcel/test/temp/Lock2Test.java
  11. 2
      update.md

9
README.md

@ -4,8 +4,9 @@ easyexcel
[![Maven central](https://maven-badges.herokuapp.com/maven-central/com.alibaba/easyexcel/badge.svg)](https://maven-badges.herokuapp.com/maven-central/com.alibaba/easyexcel) [![Maven central](https://maven-badges.herokuapp.com/maven-central/com.alibaba/easyexcel/badge.svg)](https://maven-badges.herokuapp.com/maven-central/com.alibaba/easyexcel)
[![License](http://img.shields.io/:license-apache-brightgreen.svg)](http://www.apache.org/licenses/LICENSE-2.0.html) [![License](http://img.shields.io/:license-apache-brightgreen.svg)](http://www.apache.org/licenses/LICENSE-2.0.html)
[QQ群:662022184](//shang.qq.com/wpa/qunwpa?idkey=53d9d821b0833e3c14670f007488a61e300f00ff4f1b81fd950590d90dd80f80) [QQ群: 662022184](//shang.qq.com/wpa/qunwpa?idkey=53d9d821b0833e3c14670f007488a61e300f00ff4f1b81fd950590d90dd80f80)
[钉钉群:21960511](https://qr.dingtalk.com/action/joingroup?code=v1,k1,cchz6k12ci9B08NNqhNRFGXocNVHrZtW0kaOtTKg/Rk=&_dt_no_comment=1&origin=11) [钉钉群: 21960511](https://qr.dingtalk.com/action/joingroup?code=v1,k1,cchz6k12ci9B08NNqhNRFGXocNVHrZtW0kaOtTKg/Rk=&_dt_no_comment=1&origin=11)
[官方网站: https://alibaba-easyexcel.github.io/](https://alibaba-easyexcel.github.io/)
# JAVA解析Excel工具easyexcel # JAVA解析Excel工具easyexcel
Java解析、生成Excel比较有名的框架有Apache poi、jxl。但他们都存在一个严重的问题就是非常的耗内存,poi有一套SAX模式的API可以一定程度的解决一些内存溢出的问题,但POI还是有一些缺陷,比如07版Excel解压缩以及解压后存储都是在内存中完成的,内存消耗依然很大。easyexcel重写了poi对07版Excel的解析,能够原本一个3M的excel用POI sax依然需要100M左右内存降低到几M,并且再大的excel不会出现内存溢出,03版依赖POI的sax模式。在上层做了模型转换的封装,让使用者更加简单方便 Java解析、生成Excel比较有名的框架有Apache poi、jxl。但他们都存在一个严重的问题就是非常的耗内存,poi有一套SAX模式的API可以一定程度的解决一些内存溢出的问题,但POI还是有一些缺陷,比如07版Excel解压缩以及解压后存储都是在内存中完成的,内存消耗依然很大。easyexcel重写了poi对07版Excel的解析,能够原本一个3M的excel用POI sax依然需要100M左右内存降低到几M,并且再大的excel不会出现内存溢出,03版依赖POI的sax模式。在上层做了模型转换的封装,让使用者更加简单方便
@ -15,11 +16,9 @@ Java解析、生成Excel比较有名的框架有Apache poi、jxl。但他们都
![img](img/readme/large.png) ![img](img/readme/large.png)
## 相关文档 ## 相关文档
* [快速使用](/quickstart.md) * [快速使用](https://alibaba-easyexcel.github.io/)
* [关于软件](/abouteasyexcel.md) * [关于软件](/abouteasyexcel.md)
* [常见问题](/problem.md)
* [更新记事](/update.md) * [更新记事](/update.md)
* [English-README](/easyexcel_en.md)
## 维护者 ## 维护者
姬朋飞(玉霄) 姬朋飞(玉霄)

5
docs/API.md

@ -12,6 +12,7 @@
* `ExcelIgnore` 默认所有字段都会和excel去匹配,加了这个注解会忽略该字段 * `ExcelIgnore` 默认所有字段都会和excel去匹配,加了这个注解会忽略该字段
* `DateTimeFormat` 日期转换,用`String`去接收excel日期格式的数据会调用这个注解。里面的`value`参照`java.text.SimpleDateFormat` * `DateTimeFormat` 日期转换,用`String`去接收excel日期格式的数据会调用这个注解。里面的`value`参照`java.text.SimpleDateFormat`
* `NumberFormat` 数字转换,用`String`去接收excel数字格式的数据会调用这个注解。里面的`value`参照`java.text.DecimalFormat` * `NumberFormat` 数字转换,用`String`去接收excel数字格式的数据会调用这个注解。里面的`value`参照`java.text.DecimalFormat`
* `ExcelIgnoreUnannotated` 默认不加`ExcelProperty` 的注解的都会参与读写,加了不会参与
### 参数 ### 参数
#### 通用参数 #### 通用参数
`ReadWorkbook`,`ReadSheet` 都会有的参数,如果为空,默认使用上级。 `ReadWorkbook`,`ReadSheet` 都会有的参数,如果为空,默认使用上级。
@ -21,6 +22,7 @@
* `head` 与`clazz`二选一。读取文件头对应的列表,会根据列表匹配数据,建议使用class。 * `head` 与`clazz`二选一。读取文件头对应的列表,会根据列表匹配数据,建议使用class。
* `clazz` 与`head`二选一。读取文件的头对应的class,也可以使用注解。如果两个都不指定,则会读取全部数据。 * `clazz` 与`head`二选一。读取文件的头对应的class,也可以使用注解。如果两个都不指定,则会读取全部数据。
* `autoTrim` 字符串、表头等数据自动trim * `autoTrim` 字符串、表头等数据自动trim
* `password` 读的时候是否需要使用密码
#### ReadWorkbook(理解成excel对象)参数 #### ReadWorkbook(理解成excel对象)参数
* `excelType` 当前excel的类型 默认会自动判断 * `excelType` 当前excel的类型 默认会自动判断
* `inputStream` 与`file`二选一。读取文件的流,如果接收到的是流就只用,不用流建议使用`file`参数。因为使用了`inputStream` easyexcel会帮忙创建临时文件,最终还是`file` * `inputStream` 与`file`二选一。读取文件的流,如果接收到的是流就只用,不用流建议使用`file`参数。因为使用了`inputStream` easyexcel会帮忙创建临时文件,最终还是`file`
@ -36,6 +38,7 @@
* `ExcelIgnore` 默认所有字段都会写入excel,这个注解会忽略这个字段 * `ExcelIgnore` 默认所有字段都会写入excel,这个注解会忽略这个字段
* `DateTimeFormat` 日期转换,将`Date`写到excel会调用这个注解。里面的`value`参照`java.text.SimpleDateFormat` * `DateTimeFormat` 日期转换,将`Date`写到excel会调用这个注解。里面的`value`参照`java.text.SimpleDateFormat`
* `NumberFormat` 数字转换,用`Number`写excel会调用这个注解。里面的`value`参照`java.text.DecimalFormat` * `NumberFormat` 数字转换,用`Number`写excel会调用这个注解。里面的`value`参照`java.text.DecimalFormat`
* `ExcelIgnoreUnannotated` 默认不加`ExcelProperty` 的注解的都会参与读写,加了不会参与
### 参数 ### 参数
#### 通用参数 #### 通用参数
`WriteWorkbook`,`WriteSheet` ,`WriteTable`都会有的参数,如果为空,默认使用上级。 `WriteWorkbook`,`WriteSheet` ,`WriteTable`都会有的参数,如果为空,默认使用上级。
@ -53,6 +56,8 @@
* `templateInputStream` 模板的文件流 * `templateInputStream` 模板的文件流
* `templateFile` 模板文件 * `templateFile` 模板文件
* `autoCloseStream` 自动关闭流。 * `autoCloseStream` 自动关闭流。
* `password` 写的时候是否需要使用密码
* `useDefaultStyle` 写的时候是否是使用默认头
#### WriteSheet(就是excel的一个Sheet)参数 #### WriteSheet(就是excel的一个Sheet)参数
* `sheetNo` 需要写入的编码。默认0 * `sheetNo` 需要写入的编码。默认0
* `sheetName` 需要些的Sheet名称,默认同`sheetNo` * `sheetName` 需要些的Sheet名称,默认同`sheetNo`

116
problem.md

@ -1,116 +0,0 @@
# 常见问题汇总
## 1、系统环境变量缺失或JDK版本不支持
```
java.lang.NullPointerException
at sun.awt.FontConfiguration.getVersion(FontConfiguration.java:1264)
at sun.awt.FontConfiguration.readFontConfigFile(FontConfiguration.java:219)
at sun.awt.FontConfiguration.init(FontConfiguration.java:107)
at sun.awt.X11FontManager.createFontConfiguration(X11FontManager.java:774)
at sun.font.SunFontManager$2.run(SunFontManager.java:431)
at java.security.AccessController.doPrivileged(Native Method)
at sun.font.SunFontManager.<init>(SunFontManager.java:376)
at sun.awt.FcFontManager.<init>(FcFontManager.java:35)
at sun.awt.X11FontManager.<init>(X11FontManager.java:57)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at sun.font.FontManagerFactory$1.run(FontManagerFactory.java:83)
at java.security.AccessController.doPrivileged(Native Method)
at sun.font.FontManagerFactory.getInstance(FontManagerFactory.java:74)
at java.awt.Font.getFont2D(Font.java:495)
at java.awt.Font.canDisplayUpTo(Font.java:2080)
at java.awt.font.TextLayout.singleFont(TextLayout.java:470)
at java.awt.font.TextLayout.<init>(TextLayout.java:531)
at org.apache.poi.ss.util.SheetUtil.getDefaultCharWidth(SheetUtil.java:275)
at org.apache.poi.xssf.streaming.AutoSizeColumnTracker.<init>(AutoSizeColumnTracker.java:117)
at org.apache.poi.xssf.streaming.SXSSFSheet.<init>(SXSSFSheet.java:79)
at org.apache.poi.xssf.streaming.SXSSFWorkbook.createAndRegisterSXSSFSheet(SXSSFWorkbook.java:656)
at org.apache.poi.xssf.streaming.SXSSFWorkbook.createSheet(SXSSFWorkbook.java:677)
at org.apache.poi.xssf.streaming.SXSSFWorkbook.createSheet(SXSSFWorkbook.java:83)
at com.alibaba.excel.write.context.GenerateContextImpl.buildCurrentSheet(GenerateContextImpl.java:93)
at com.alibaba.excel.write.ExcelBuilderImpl.addContent(ExcelBuilderImpl.java:53)
at com.alibaba.excel.ExcelWriter.write(ExcelWriter.java:44)
```
### 解决方法
该异常由于自己的环境变量缺少swing需要的字体配置,检查自己应用是否有配置-Djava.awt.headless=true,如果没有请加上该系统参数,可以解决问题。如果仍旧不行,在dockerfile中增加字体安装命令:
![粘贴图片.png](http://ata2-img.cn-hangzhou.img-pub.aliyun-inc.com/a857edfbc8199db7bb35b9e99f1f57d5.png)
参考:
https://lark.alipay.com/aone355606/gfqllg/ulptif
https://stackoverflow.com/questions/30626136/cannot-load-font-in-jre-8 http://www.jianshu.com/p/c05b5fc71bd0
## 2、xls格式错用xlsx方式解析
```
org.apache.poi.openxml4j.exceptions.OLE2NotOfficeXmlFileException: The supplied data appears to be in the OLE2 Format. You are calling the part of POI that deals with OOXML (Office Open XML) Documents. You need to call a different part of POI to process this data (eg HSSF instead of XSSF)
at org.apache.poi.openxml4j.opc.internal.ZipHelper.verifyZipHeader(ZipHelper.java:172)
at org.apache.poi.openxml4j.opc.internal.ZipHelper.openZipStream(ZipHelper.java:229)
at org.apache.poi.openxml4j.opc.ZipPackage.<init>(ZipPackage.java:97)
at org.apache.poi.openxml4j.opc.OPCPackage.open(OPCPackage.java:342)
at com.alibaba.excel.read.v07.XlsxSaxAnalyser.<init>(XlsxSaxAnalyser.java:46)
at com.alibaba.excel.read.ExcelAnalyserImpl.getSaxAnalyser(ExcelAnalyserImpl.java:30)
at com.alibaba.excel.read.ExcelAnalyserImpl.analysis(ExcelAnalyserImpl.java:51)
at com.alibaba.excel.ExcelReader.read(ExcelReader.java:55)
at read.v07.Read2007Xlsx.noModel(Read2007Xlsx.java:42)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at mockit.integration.junit4.internal.BlockJUnit4ClassRunnerDecorator.executeTest(BlockJUnit4ClassRunnerDecorator.java:126)
at mockit.integration.junit4.internal.BlockJUnit4ClassRunnerDecorator.invokeExplosively(BlockJUnit4ClassRunnerDecorator.java:104)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:237)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
```
### 解决方法
该异常时由于03版的xls,文件用07版的方式做解析的报错,请检查excelType是否设置错误。或者是不是手动去修改了excel文件名后缀的xls为xlsx。
## 3、xlsx错用xls解析
```
org.apache.poi.poifs.filesystem.OfficeXmlFileException: The supplied data appears to be in the Office 2007+ XML. You are calling the part of POI that deals with OLE2 Office Documents. You need to call a different part of POI to process this data (eg XSSF instead of HSSF)
at org.apache.poi.poifs.storage.HeaderBlock.<init>(HeaderBlock.java:152)
at org.apache.poi.poifs.storage.HeaderBlock.<init>(HeaderBlock.java:140)
at org.apache.poi.poifs.filesystem.NPOIFSFileSystem.<init>(NPOIFSFileSystem.java:302)
at org.apache.poi.poifs.filesystem.POIFSFileSystem.<init>(POIFSFileSystem.java:87)
at com.alibaba.excel.read.v03.XlsSaxAnalyser.<init>(XlsSaxAnalyser.java:55)
at com.alibaba.excel.read.ExcelAnalyserImpl.getSaxAnalyser(ExcelAnalyserImpl.java:27)
at com.alibaba.excel.read.ExcelAnalyserImpl.analysis(ExcelAnalyserImpl.java:51)
at com.alibaba.excel.ExcelReader.read(ExcelReader.java:55)
at read.v03.XLS2003FunctionTest.testExcel2003NoModel(XLS2003FunctionTest.java:31)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at mockit.integration.junit3.internal.JUnitTestCaseDecorator.runTest(JUnitTestCaseDecorator.java:142)
at mockit.integration.junit3.internal.JUnitTestCaseDecorator.originalRunBare(JUnitTestCaseDecorator.java:102)
at mockit.integration.junit3.internal.JUnitTestCaseDecorator.runBare(JUnitTestCaseDecorator.java:87)
```
原理和原因大致同上

1365
quickstart.md

File diff suppressed because it is too large Load Diff

2
src/main/java/com/alibaba/excel/analysis/ExcelAnalyserImpl.java

@ -60,7 +60,7 @@ public class ExcelAnalyserImpl implements ExcelAnalyser {
} }
switch (excelType) { switch (excelType) {
case XLS: case XLS:
POIFSFileSystem poifsFileSystem = null; POIFSFileSystem poifsFileSystem;
if (readWorkbookHolder.getFile() != null) { if (readWorkbookHolder.getFile() != null) {
poifsFileSystem = new POIFSFileSystem(readWorkbookHolder.getFile()); poifsFileSystem = new POIFSFileSystem(readWorkbookHolder.getFile());
} else { } else {

10
src/main/java/com/alibaba/excel/read/metadata/holder/ReadWorkbookHolder.java

@ -141,11 +141,11 @@ public class ReadWorkbookHolder extends AbstractReadHolder {
} else { } else {
this.autoCloseStream = readWorkbook.getAutoCloseStream(); this.autoCloseStream = readWorkbook.getAutoCloseStream();
} }
if (readWorkbook.getExcelType() == null) {
this.excelType = ExcelTypeEnum.valueOf(file, inputStream); // The type of excel is read according to the judgment.Because encrypted XLSX needs to be specified as XLS to
} else { // properly parse.
this.excelType = readWorkbook.getExcelType(); this.excelType = ExcelTypeEnum.valueOf(file, inputStream, readWorkbook.getExcelType());
}
if (ExcelTypeEnum.XLS == excelType && getGlobalConfiguration().getUse1904windowing() == null) { if (ExcelTypeEnum.XLS == excelType && getGlobalConfiguration().getUse1904windowing() == null) {
getGlobalConfiguration().setUse1904windowing(Boolean.FALSE); getGlobalConfiguration().setUse1904windowing(Boolean.FALSE);
} }

8
src/main/java/com/alibaba/excel/support/ExcelTypeEnum.java

@ -29,7 +29,7 @@ public enum ExcelTypeEnum {
this.setValue(value); this.setValue(value);
} }
public static ExcelTypeEnum valueOf(File file, InputStream inputStream) { public static ExcelTypeEnum valueOf(File file, InputStream inputStream, ExcelTypeEnum excelType) {
try { try {
FileMagic fileMagic; FileMagic fileMagic;
if (file != null) { if (file != null) {
@ -59,9 +59,15 @@ public enum ExcelTypeEnum {
return XLSX; return XLSX;
} }
} catch (IOException e) { } catch (IOException e) {
if (excelType != null) {
return excelType;
}
throw new ExcelCommonException( throw new ExcelCommonException(
"Convert excel format exception.You can try specifying the 'excelType' yourself", e); "Convert excel format exception.You can try specifying the 'excelType' yourself", e);
} }
if (excelType != null) {
return excelType;
}
throw new ExcelCommonException( throw new ExcelCommonException(
"Convert excel format exception.You can try specifying the 'excelType' yourself"); "Convert excel format exception.You can try specifying the 'excelType' yourself");
} }

2
src/main/java/com/alibaba/excel/write/metadata/WriteWorkbook.java

@ -43,7 +43,7 @@ public class WriteWorkbook extends WriteBasicParameter {
*/ */
private File templateFile; private File templateFile;
/** /**
* Default true * Default trueuseDefaultStyle
*/ */
private Boolean autoCloseStream; private Boolean autoCloseStream;
/** /**

6
src/test/java/com/alibaba/easyexcel/test/core/encrypt/EncryptDataTest.java

@ -57,7 +57,7 @@ public class EncryptDataTest {
@Test @Test
public void t02ReadAndWriteStream03() throws Exception { public void t02ReadAndWriteStream03() throws Exception {
readAndWriteStream(file03OutputStream, ExcelTypeEnum.XLSX); readAndWriteStream(file03OutputStream, ExcelTypeEnum.XLS);
} }
private void readAndWrite(File file) { private void readAndWrite(File file) {
@ -72,8 +72,8 @@ public class EncryptDataTest {
fileOutputStream.close(); fileOutputStream.close();
FileInputStream fileInputStream = new FileInputStream(file); FileInputStream fileInputStream = new FileInputStream(file);
EasyExcel.read(fileInputStream, EncryptData.class, new EncryptDataListener()).password("123456").sheet() EasyExcel.read(fileInputStream, EncryptData.class, new EncryptDataListener()).password("123456")
.doRead(); .excelType(excelType).sheet().doRead();
} }
private List<SimpleData> data() { private List<SimpleData> data() {

2
src/test/java/com/alibaba/easyexcel/test/temp/Lock2Test.java

@ -29,7 +29,7 @@ public class Lock2Test {
@Test @Test
public void test() throws Exception { public void test() throws Exception {
File file = new File("D:\\test\\test001.xlsx"); File file = new File("D:\\test\\000001.xlsx");
List<Object> list = EasyExcel.read(file).sheet().headRowNumber(0).doReadSync(); List<Object> list = EasyExcel.read(file).sheet().headRowNumber(0).doReadSync();
LOGGER.info("数据:{}", list.size()); LOGGER.info("数据:{}", list.size());

2
update.md

@ -2,7 +2,7 @@
* 新增支持导入、导出支持公式 * 新增支持导入、导出支持公式
* 新增支持读取单元格类型、写入指定单元格类型 * 新增支持读取单元格类型、写入指定单元格类型
* 支持通过模板填充数据 * 支持通过模板填充数据
* 新增写支持 禁用头样式 `autoHeadStyle` * 新增写支持 禁用头样式 `useDefaultStyle`
* 用map读取数据 空的单元格也会有个 null的数据 * 用map读取数据 空的单元格也会有个 null的数据
* 转换报错 能获取到对应的行号和列号 * 转换报错 能获取到对应的行号和列号
* 优化读取全部sheet方案 * 优化读取全部sheet方案

Loading…
Cancel
Save