前言:FastDFS是什么呢?
FastDFS是一个开源的轻量级分布式文件系统,它对文件进行管理,功能包括:文件存储、文件同步、文件访问(文件上传、文件下载)等,解决了大容量存储和负载均衡的问题。特别适合以文件为载体的在线服务,如相册网站、视频网站等等。
FastDFS为互联网量身定制,充分考虑了冗余备份、负载均衡、线性扩容等机制,并注重高可用、高性能等指标,使用FastDFS很容易搭建一套高性能的文件服务器集群提供文件上传、下载等服务。
文章目录一。基础配置二。文件上传三。文件下载四。扩展 一。基础配置1.在pom.xml文件中导入依赖:
org.csource fastdfs-client-java 1.29-SNAPSHOT注意:该依赖在maven中心仓库中并不存在,需要手动打包并添加到类库中,具体操作方法参照我的另外一篇博客:解决Dependency ‘org.csource:fastdfs-client-java:1.29-SNAPSHOT‘ not found(解决maven仓库中找不到依赖)的问题
2.FastDFS配置文件fdfs_client.conf,其中主要是对连接FastDFS进行一些配置:
#连接超时时间connect_timeout = 60#网络超时时间network_timeout = 60#字符编码charset = UTF-8#端口http.tracker_http_port = 7099http.anti_steal_token = no#ip地址tracker_server = 192.168.23.222:221223.连接FastDFS工具类FastDFSClient ,其中包含了读取FastDFS的配置文件,文件上传与下载等方法:
import org.csource.common.MyException;import org.csource.common.NameValuePair;import org.csource.fastdfs.*;import org.slf4j.Logger;import org.slf4j.LoggerFactory;import org.springframework.stereotype.Component;import java.io.IOException;public class FastDFSClient {/** * it is to define an logger to record the operation's log */private static Logger logger = LoggerFactory.getLogger(FastDFSClient.class);static {try {//线上//String filePath = "/home/data/config/fdfs_client.conf";//线下String filePath = "D:\\data\\config\\fdfs_client.conf";ClientGlobal.init(filePath);//String filePath = new ClassPathResource("fdfs_client.pr operties").getFile().getAbsolutePath();//ClientGlobal.initByProperties(filePath);} catch (IOException e) {logger.error("Get FastDFS config file fail!",e);} catch (MyException e) {logger.error("FastDF init fail!",e);}}/** * 根据文件名,文件类型以及字节流数组上传文件 * @param fileName * @param fileExt * @param fileBuff * @return */public static String[] upload(String fileName,String fileExt,byte[] fileBuff) {logger.info("File Name : " + fileName + ",File Length : " + fileBuff.length);NameValuePair[] meta_list = new NameValuePair[1];meta_list[0] = new NameValuePair("author","root");long startTime = System.currentTimeMillis();String[] uploadResults = null;StorageClient storageClient = null;try {storageClient = getTrackerClient();uploadResults = storageClient.upload_file(fileBuff,fileExt,meta_list);} catch (IOException e) {logger.error("IO Exception when uploading the file : " + fileName,e);} catch (MyException e) {logger.error("Non IO Exception when uploading the file : " + fileName,e);}logger.info("upload_file time used : " + (System.currentTimeMillis() - startTime) + "ms");if (uploadResults == null && storageClient != null) {logger.error("upload file fail, error code : " + storageClient.getErrorCode());} else {String groupName = uploadResults[0];String remoteFileName = uploadResults[1];logger.info("upload file successfully! " + "group_name : " + groupName + ", remoteFileName : " + remoteFileName );}return uploadResults;}/** * @description this function is to download file from FastDFS and return the byte stream * @param groupName * @param remoteName * @return */public static byte[] downloadFile(String groupName, String remoteName) {byte[] fileByte = null;try {StorageClient storageClient = getTrackerClient();System.out.println("group name:"+groupName + ", remote name:" + remoteName);fileByte = storageClient.download_file(groupName,remoteName);} catch (IOException e) {logger.error("IO Exception: Get file from FastDFS Server failed!",e);} catch (MyException e) {logger.error("Non IO Exception: Get file from FastDFS Server failed!",e);}return fileByte;}/** * @description this function is to delete appointed file that stored at FastDFS * @param groupName * @param remoteName * @return */public static Integer deleteFile(String groupName,String remoteName) {Integer deleteFileNums = null;try {StorageClient storageClient = getTrackerClient();deleteFileNums = storageClient.delete_file(groupName,remoteName);} catch (IOException e) {logger.error("IO Exception: Delete file failed!",e);} catch (MyException e) {logger.error("Non IO Exception: Delete file failed",e);}return deleteFileNums;}/** * @description this function is to get appointed file's info from FastDFS * @param groupName * @param remoteName * @return */public static FileInfo getFileInfo(String groupName, String remoteName) {try {StorageClient storageClient = getTrackerClient();return storageClient.get_file_info(groupName,remoteName);} catch (IOException e) {logger.error("IO Exception: Get FileInfo from FastDFS failed!",e);} catch (MyException e) {logger.error("Non IO Exception: Get FileInfo from FastDFS failed!",e);}return null;}/** * @param groupName * @return * @throws IOException */public static StorageServer[] getStoreStorages(String groupName) throws IOException, MyException {TrackerClient trackerClient = new TrackerClient();TrackerServer trackerServer = trackerClient.getConnection();return trackerClient.getStoreStorages(trackerServer,groupName);}/** * @param groupName * @param remoteFileName * @return * @throws IOException */public static ServerInfo[] getFetchStorages(String groupName,String remoteFileName) throws IOException, MyException {TrackerClient trackerClient = new TrackerClient();TrackerServer trackerServer = trackerClient.getConnection();return trackerClient.getFetchStorages(trackerServer,groupName,remoteFileName);}public static String getFilePath(String groupName, String remoteFileName) throws IOException {//线上//String filePath = "http://218.192.110.129:7099/" + groupName + "/" +remoteFileName;//本地String filePath = getTrackerUrl() + groupName + "/" +remoteFileName;System.out.println("filePath: " + filePath);return filePath;}/** * @return * @throws IOException */private static String getTrackerUrl() throws IOException {return "http://" + getTrackerServer().getInetSocketAddress().getHostString() + ":" + ClientGlobal.getG_tracker_http_port() + "/";}/** * @description this function is to get the FastDFS's storageClient * @return * @throws IOException */private static StorageClient getTrackerClient() throws IOException {TrackerServer trackerServer = getTrackerServer();StorageClient storageClient = new StorageClient(trackerServer,null);return storageClient;}/** * @description this function is to get the FastDFS's trackerClient * @return * @throws IOException */private static TrackerServer getTrackerServer() throws IOException {TrackerClient trackerClient = new TrackerClient();TrackerServer trackerServer = trackerClient.getConnection();return trackerServer;}} 二。文件上传1.文件上传的主要实现思路是将上传文件转换成byte数组,获取文件上传名称以及文件类型,调用StorageClient类中的upload_file方法,该方法执行成功后返回上传的所属组别以及远程文件名称,再用/连接得到访问url,控制器上传接口代码如下:
/** * 发送个人站内信 * * @param jwt * @param toIds * @param title * @param content * @param multipartFile * @return */@ApiOperation(value = "发送个人站内信")@PostMapping("/send")public HttpRespond send(@RequestHeader("Authorization") String jwt,@RequestParam(value = "toIds") Integer[] toIds,@RequestParam(value = "title") String title,@RequestParam(value = "content") String content,@RequestParam(value = "multipartFile") MultipartFile multipartFile) {// 获取当前登录的用户信息: userId,HashMap userInfo = JsonInfoUtils.getUserInfo(jwt);if (userInfo == null) {return HttpRespond.unauthorized();}Integer userId = Integer.valueOf(userInfo.get("userId"));// 接收人不能为空if (toIds == null || toIds.length 100){return "wrongFileSize";}String size = fileSize + "B";String[] uploadResults = null;byte[] fileBuff = null;InputStream inputStream = null;try {//接受字符文件输入流inputStream = multipartFile.getInputStream();if (inputStream != null) {//获取数据流中含有字节数int len1 = inputStream.available();fileBuff = new byte[len1];//从(来源)输入流中(读取内容)读取的一定数量字节数,并将它们存储到(去处)缓冲区数组fileBuff中inputStream.read(fileBuff);}} catch (IOException e) {log.error("IO Exception : add file exception!", e);} finally {try {inputStream.close();} catch (IOException e) {log.error("IO Exception : close Input Stream error!", e);}}//上传文件,返回所属组别以及远程文件名称uploadResults = FastDFSClient.upload(fileName, fileExt, fileBuff);String groupName = uploadResults[0];String remoteName = uploadResults[1];String url = null;try {//得到上传后远程文件的访问路径url = FastDFSClient.getFilePath(groupName, remoteName);} catch (IOException e) {log.error("IO Exception : close Input Stream error!", e);}//封装数据MessageAttachment attachment = new MessageAttachment();attachment.setTargetId(targetId);attachment.setTargetType(targetType);attachment.setOriginalName(fileOriName);attachment.setActualName(fileName);attachment.setSize(size);attachment.setUrl(url);attachment.setFileType(fileExt);attachmentService.save(attachment);return "success";}2.返回上传文件大小工具类:
/** * @Description:返回上传文件大小 * @Author :zks * @Date :9:55 2020/8/20 */public class FileSizeUtil {/** * length:文件长度(multipartFile.getSize()) * unit限制单位(B,K,M,G) * @param length * @param unit * @return */public static Double getFileSize(long length,String unit){double fileSize = 0;if ("B".equals(unit.toUpperCase())) {fileSize = (double) length;} else if ("K".equals(unit.toUpperCase())) {fileSize = (double) length / 1024;} else if ("M".equals(unit.toUpperCase())) {fileSize = (double) length / 1048576;} else if ("G".equals(unit.toUpperCase())) {fileSize = (double) length / 1073741824;}return fileSize;}}3.使用postman进行测试:
4.数据库表结构如下:
5.插入的数据如下:
16 19 1 1575635185311.jpg 1575635185311 84924.0B http://192.168.23.222:7099/group1/M00/00/02/wKgX3l9DjXmAUiFeAAFLvP-tZBQ642.jpg jpg 2020-08-24 17:50:49 2020-08-24 17:50:49 0
6.根据数据库表信息查看上传文件,路径为:http://192.168.23.222:7099/group1/M00/00/02/wKgX3l9DjXmAUiFeAAFLvP-tZBQ642.jpg
注意:该访问地址由ip+端口+上传组别+fastdfs内部定义上传位置+内部定义上传名称
三。文件下载1.文件下载主要实现思路是从数据库拿到访问该文件的远程url,再以/将其进行切割,得到该文件所在组名以及文件地址,调用StorageClient类中的download_file方法,该方法执行成功返回该文件的字节数组。先根据接口传进来的下载位置参数,以及数据库表存储的文件名称生成新文件,再调用FileUtils类中的writeByteArrayToFile方法将返回的byte数组写入到新文件中,控制器下载接口代码如下:
/** * 附件下载 * * @param id * @param location * @return * @throws IOException */@ApiOperation(value = "根据附件id下载附件")@GetMapping("/download")public HttpRespond download(@RequestParam("id") Integer id, @RequestParam("location") String location) throws IOException {//根据id查找附件信息MessageAttachment messageAttachment = attachmentService.getById(id);// http://192.168.23.222:7099/group1/M00/00/02/wKgX3l8-F9KABdUlAAP5BN0hxz4168.pdfString s = messageAttachment.getUrl();//以“/”切割4次String[] ss = s.split("/", 5);//根据fastdfs上传组别以及资源名称得到下载字节数组 ss[3]:group1 ss[4]:M00/00/02/wKgX3l8-F9KABdUlAAP5BN0hxz4168.pdfbyte[] bytes = FastDFSClient.downloadFile(ss[3], ss[4]);// 给定文件路径和名称,把"\"替换成"/",target为转义字符,注意替换后重新给字符串赋值//D:\fastdfsDownload ----->D:/fastdfsDownloadlocation = location.replace("\\", "/");String pathName = location + "/" + messageAttachment.getOriginalName();File file = new File(pathName);//将字节流数组写入文件FileUtils.writeByteArrayToFile(file, bytes);return HttpRespond.success(pathName);}2.使用postman测试该下载接口:
注意:上面我用红圈标注的传入参数格式你是不是觉得很奇怪?其实那就是一个下载地址,当我们传入参数中带有+,空格,/,?,%,#,&,=等特殊符号的时候,可能在服务器端无法获得正确的参数值,此时需要对字符进行转译。这里我们对:以及\进行了转译,对应转译字符分别为%3A和%2F。
3.打开该文件目录,文件已存在:
1.可能用到过FastDFS的小伙伴们都想过一个问题,上传文件的文件名能更改吗?更改之后通过更改后的url能对其进行访问吗?现在我们来试一试,首先登陆服务器,找到我们上传的文件:
2.修改文件名为:football.jpg:
3.使用url加上修改后的文件名进行访问:
4.结果当然是访问出错,找不到该文件,因为上传到fastdfs的文件,它会依照其内部的命名逻辑来进行对应的命名,只有经过其内部逻辑进行命名后,fastdfs才能找到该相应的文件,如果将文件名进行修改,则会出现找不到文件的问题。