Java - IO

Updated: 2019-01-03

New IO

Since Java 7.

Key new classes: Path, Paths, FileSystem, FileSystems

Path path = Paths.get("/path/to")

Path path = FileSystems.getDefault().getPath(".", name);

Read all of the bytes in the file:

byte[] filearray = Files.readAllBytes(path);

Read all of the lines in the file:

List<String> lines = Files.readAllLines(path, Charset.defaultCharset());

Read by buffered IO:

BufferedReader reader = Files.newBufferedReader(path, Charset.defaultCharset() );
String line = null;
while ((line = reader.readLine()) != null) {
    // ...
}

Write to file

String content =// create new, overwrite if exists
Files.write( path, content.getBytes(), StandardOpenOption.CREATE);

Buffered writer:

BufferedWriter writer = Files.newBufferedWriter(path, Charset.defaultCharset(),
                                              StandardOpenOption.CREATE);
writer.write(content, 0, content.length());

Create

Files.write(Paths.get("/path/to/file"), Arrays.asList("content"));

Append

Files.write(Paths.get("/path/to/file"), Arrays.asList("content"), StandardOpenOption.APPEND);

Byte Streams

  • InputStream: FileInputStream
  • OutputStream: FileOutputStream
public static void main(String[] args) throws IOException {
    FileInputStream in = null;
    FileOutputStream out = null;

    try {
        in = new FileInputStream("xanadu.txt");
        out = new FileOutputStream("outagain.txt");
        int c;

        while ((c = in.read()) != -1) {
            out.write(c);
        }
    } finally {
        if (in != null) {
            in.close();
        }
        if (out != null) {
            out.close();
        }
    }
}

Character Streams

  • Reader: FileReader
  • Writer: FileWriter
public static void main(String[] args) throws IOException {

    FileReader inputStream = null;
    FileWriter outputStream = null;

    try {
        inputStream = new FileReader("xanadu.txt");
        outputStream = new FileWriter("characteroutput.txt");

        int c;
        while ((c = inputStream.read()) != -1) {
            outputStream.write(c);
        }
    } finally {
        if (inputStream != null) {
            inputStream.close();
        }
        if (outputStream != null) {
            outputStream.close();
        }
    }
}

Byte to Character

  • InputStreamReader
  • OutputStreamWriter

Line-Oriented

  • BufferedReader
  • PrintWriter
BufferedReader in = new BufferedReader(new FileReader(filename))

PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter(file)));
PrintWriter out = new PrintWriter(file);

Buffered Streams

  • BufferedInputStream
  • BufferedOutputStream
  • BufferedReader
  • BufferedWriter

Scanner

public static void main(String[] args) throws IOException {
    Scanner s = null;

    try {
        s = new Scanner(new BufferedReader(new FileReader("xanadu.txt")));

        while (s.hasNext()) {
            System.out.println(s.next());
        }
    } finally {
        if (s != null) {
            s.close();
        }
    }
}

Redirect

PrintStream console = System.out;

BufferedInputStream in = new BufferedInputStream(
    new FileInputStream("Redirecting.java"));

PrintStream out = new PrintStream(
    new BufferedOutputStream(
    new FileOutputStream("test.out")));

System.setIn(in);
System.setOut(out);
System.setErr(out);

Misc.

  • Data written by DataOutputStream is guaranteed to be accurately recovered by DataInputStream
  • System.out is a PrintStream, which is an OutputStream
  • Convert system.out to PrintWriter. true to enable automatic flushing.
PrintWriter out = new PrintWriter(System.out, true);

Copy From Local

Configuration conf = new Configuration();
conf.addResource(new Path(pathHadoopCoreSite));
conf.addResource(new Path(pathHadoopHDFSSite));
FileSystem fs = FileSystem.get(conf);

Path src = new Path(pathLocal);
Path dst = new Path(pathHDFS);

fs.copyFromLocalFile(src, dst);

Work on Directories

Local:

File dir = new File(pathStr);
for (File file : dir.listFiles()) {
    if (file.getName().startsWith("part-")) {
        scanner = new Scanner(new BufferedReader(new FileReader(file)));
        // do something ...
    }
}

Redirect

Stream st = new Stream(new FileOutputStream("output.txt"));
System.setErr(st);
System.setOut(st);

Read Twice

in = fs.open(new Path(uri));
IOUtils.copyBytes(in, System.out, 4096, false);
in.seek(0); // go back to the start of the file
IOUtils.copyBytes(in, System.out, 4096, false);

Why BufferedWriter is not Writing

There are a few ways to create a BufferedWriter, e.g. by Files.newBufferedWriter

BufferedWriter writer = Files.newBufferedWriter(Paths.get("filename"), StandardCharsets.UTF_8, StandardOpenOption.CREATE);

or by new

BufferedWriter writer = new BufferedWriter(new FileWriter("filename"));

you may try to call writer.write() to dump some text to the file

writer.write("a line of text");

the program executes fine, but there's nothing in the file.

The reason is that BufferedWriter needs a flush to dump the buffered text to file:

writer.flush()

Or this may happen when you close the writer

writer.close()

Checkout the JavaDoc for more details.