Java Concurrent Animated

Java Concurrent Animated

Victor Grazi дуже просто розказав про складний java.util.concurrent. 

Показав як і для чого використовуються  

• Executors
• Future
• synchronized
• ReentrantLock
• Condition
• Semaphore
• ReadWriteLock
• CountDownLatch
• CyclicBarrier
• Phaser
• BlinkingPhaser
• AtomicInteger
• BlockingQueue
• TransferQueue
• CompletionService
• ConcurrentHashMap
• Fork/Join

Для презентації використана програма візуалізації, яку можна завантажити на його сайті - дуже зручно тим хто хоче ознайомитись з можливостями конкурентого API в Java чи має складності з розумінням. 

HighLoad FileDistributor: Part 1 - FileCacheAdapter

On previous meetup we discussed theoretical issue: How to develop application which allows to download few dozens small files from server, write some logs about it and execute http call if download was successful?
Actually task is quite easy but... we have few restrictions
1) We have only one Tomcat (means no cluster)
2) Expected number of downloads about 10 000 000  per day

So together we decided
1) Put files in cache
2) Execute async call to remote server
Pretty simple.

So I started implementing solution. First question was how to put file in cache. I didn't find any implemented  solution and only one article.
Below my version of caching files
 

import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;

public class FileCacheAdapter extends CacheAdapter {
  private static final Logger log = LoggerFactory.getLogger(FileCacheAdapter.class);

  @Value(value = "#{'${file.path}'}")
  private String filePath;

  public InputStream getFile(String fileName) throws IOException {
    StringBuilder lookupFile = new StringBuilder(filePath);
    lookupFile.append("\\");
    lookupFile.append(fileName);
    Path filePath = Paths.get(lookupFile.toString());
    boolean isExist = Files.exists(filePath);
    if (!isExist) {
      log.debug("File with fileName: {} was removed!", fileName);
      remove(fileName);
      return null;
    }
    long lastModified = Files.getLastModifiedTime(filePath).toMillis();
    FileStamp fileStamp = get(fileName);
    if (fileStamp != null && fileStamp.getLastModified() == lastModified) {
      return new ByteArrayInputStream(fileStamp.getContent());
    }
    updateFile(fileName);
    return getFile(fileName);
  }
  
  private void updateFile(String fileName) throws IOException{
    remove(fileName);
    putFile(fileName);
  }

  public void putFile(String fileName) throws IOException {
    tryWriteLockOnKey(fileName, 2000L);
    try {
      if (get(fileName) != null) {
        return;
      }
      StringBuilder lookupFile = new StringBuilder(filePath);
      lookupFile.append(fileName);
      Path filePath = Paths.get(lookupFile.toString());
      byte[] fileContent = getFileContent(filePath);
      long lastModified = Files.getLastModifiedTime(filePath).toMillis();
      FileStamp fileStamp = new FileStamp(fileName, lastModified, fileContent);
      put(fileName, fileStamp);
    } finally {
      releaseWriteLockOnKey(fileName);
    }
  }

  private byte[] getFileContent(Path path) throws IOException {
    byte[] fileContent = Files.readAllBytes(path);
    return fileContent;
  }
}


public class FileStamp implements Serializable {
  /**
   * 
   */
  private static final long serialVersionUID = -4056582522653888648L;
  /**
   * A name of the file
   */
  private String fileName;
  /**
   * A time when the file was modified last time.
   */
  private long lastModified;

  /**
   * A content of the file.
   */
  private byte[] content;

  public FileStamp() {
    super();
  }

  public FileStamp(String fileName, long lastModified, byte[] content) {
    this.fileName = fileName;
    this.lastModified = lastModified;
    this.content = content;
  }

  public String getFileName() {
    return fileName;
  }

  public void setFileName(String fileName) {
    this.fileName = fileName;
  }

  public long getLastModified() {
    return lastModified;
  }

  public void setLastModified(long lastModified) {
    this.lastModified = lastModified;
  }

  public byte[] getContent() {
    return content;
  }

  public void setContent(byte[] content) {
    this.content = content;
  }

}

Lets analyze step by step how it works. Fetching file from cache

public InputStream getFile(String fileName) throws IOException {
    StringBuilder lookupFile = new StringBuilder(filePath);
    lookupFile.append("\\");
    lookupFile.append(fileName);
    Path filePath = Paths.get(lookupFile.toString());
    boolean isExist = Files.exists(filePath);
    if (!isExist) {
      log.debug("File with fileName: {} was removed!", fileName);
      remove(fileName);
      return null;
    }
    long lastModified = Files.getLastModifiedTime(filePath).toMillis();
    FileStamp fileStamp = get(fileName);
    if (fileStamp != null && fileStamp.getLastModified() == lastModified) {
      return new ByteArrayInputStream(fileStamp.getContent());
    }
    updateFile(fileName);
    return getFile(fileName);
  }

I use new NIO API just to avoid file lock. As you see I check if file is present on filesystem and return null if there is no such file.
Then I check if file was modified and updated cache if it is needed.

Next method, actually three methods are responsible for inserting file in cache

private void updateFile(String fileName) throws IOException{
    remove(fileName);
    putFile(fileName);
  }

  public void putFile(String fileName) throws IOException {
    tryWriteLockOnKey(fileName, 2000L);
    try {
      if (get(fileName) != null) {
        return;
      }
      StringBuilder lookupFile = new StringBuilder(filePath);
      lookupFile.append(fileName);
      Path filePath = Paths.get(lookupFile.toString());
      byte[] fileContent = getFileContent(filePath);
      long lastModified = Files.getLastModifiedTime(filePath).toMillis();
      FileStamp fileStamp = new FileStamp(fileName, lastModified, fileContent);
      put(fileName, fileStamp);
    } finally {
      releaseWriteLockOnKey(fileName);
    }
  }

  private byte[] getFileContent(Path path) throws IOException {
    byte[] fileContent = Files.readAllBytes(path);
    return fileContent;
  }

Just to avoid parallel cache updating I lock cache record

tryWriteLockOnKey(fileName, 2000L);

In the next part will be described async call to remote server

Java 7 release

Як відомо багатьом сьодні реліз сьомої джави з чим всіх вас і вітаю
Не обійшлось правда без ложки дьогтю
Читаємо уважно

From: Uwe Schindler
Date: Thu, 28 Jul 2011 23:13:36 +0200
Subject: [WARNING] Index corruption and crashes in Apache Lucene Core / Apache Solr with Java 7

Hello Apache Lucene & Apache Solr users,
Hello users of other Java-based Apache projects,

Oracle released Java 7 today. Unfortunately it contains hotspot compiler
optimizations, which miscompile some loops. This can affect code of several
Apache projects. Sometimes JVMs only crash, but in several cases, results
calculated can be incorrect, leading to bugs in applications (see Hotspot
bugs 7070134 [1], 7044738 [2], 7068051 [3]).

Apache Lucene Core and Apache Solr are two Apache projects, which are
affected by these bugs, namely all versions released until today. Solr users
with the default configuration will have Java crashing with SIGSEGV as soon
as they start to index documents, as one affected part is the well-known
Porter stemmer (see LUCENE-3335 [4]). Other loops in Lucene may be
miscompiled, too, leading to index corruption (especially on Lucene trunk
with pulsing codec; other loops may be affected, too - LUCENE-3346 [5]).

These problems were detected only 5 days before the official Java 7 release,
so Oracle had no time to fix those bugs, affecting also many more
applications. In response to our questions, they proposed to include the
fixes into service release u2 (eventually into service release u1, see [6]).
This means you cannot use Apache Lucene/Solr with Java 7 releases before
Update 2! If you do, please don't open bug reports, it is not the
committers' fault! At least disable loop optimizations using the
-XX:-UseLoopPredicate JVM option to not risk index corruptions.

Please note: Also Java 6 users are affected, if they use one of those JVM
options, which are not enabled by default: -XX:+OptimizeStringConcat or
-XX:+AggressiveOpts

It is strongly recommended not to use any hotspot optimization switches in
any Java version without extensive testing!

In case you upgrade to Java 7, remember that you may have to reindex, as the
unicode version shipped with Java 7 changed and tokenization behaves
differently (e.g. lowercasing). For more information, read
JRE_VERSION_MIGRATION.txt in your distribution package!

On behalf of the Lucene project,
Uwe

[1] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7070134
[2] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7044738
[3] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7068051
[4] https://issues.apache.org/jira/browse/LUCENE-3335
[5] https://issues.apache.org/jira/browse/LUCENE-3346
[6] http://s.apache.org/StQ

Отож будьте уважні