Developer for Developers

Articles about everything related to the software development

Tools

C++ code coverage profiling with GCC/GCOV

The coverage analysis with GCC/GCOV includes three following steps

  • instrumented application build — libraries, executable(s), and profiling artifacts (*.gcno files) are created
  • the application test run(s) — the runtime coverage statistics (*.gcda files) is collected
  • coverage statistics post-processing with GCOV/LCOV — text/HTML coverage reports are generated

Instrumented application build

To enable the instrumented compilation use GCC/G++ with --coveragee flag. The know-how here is to specify the full path to source files during the compilation in order to be able to perform a cross-profiling and ease the use of LCOV (described below).

$ g++ -c -g -O0 --coverage -o $PWD/obj/myclass.o $PWD/myclass.cpp
$ g++ -c -g -O0 --coverage -o $PWD/obj/main.o $PWD/main.cpp
$ g++ -g -O0 --coverage -o $PWD/bin/myapp $PWD/obj/*.o

The GNU Make has two useful functions to convert filenames to absolute ones: $(abspath ...) and $(relpath ...)

When compiling with --coverage flag, the *.gcno file is created in the same location, as the object file . This file is used by GCOV for post-processing application's statistics collected at runtime and contains profiling arcs information.

$ ls $PWD/obj
main.gcno main.o myclass.gcno myclass.o
$

Coverage statistics collection

An instrumented application collects coverage statistics at runtime and creates a set of *.gcda files (or updates existing ones) on exit. For every *.gcno file created during the build a corresponding *.gcda file is created by the instrumented application upon exit. For *.gcda files to be generated, the application must exit cleanly by either returning from main() or by calling exit().
For client/server applications I typically install SIGTERM handler to ensure a clean application termination.

The directory, where *.gcda files are to be created should exist and be writable by the application. A *.gcda file is created by default in the directory where the corresponding *.gcno file was created during the build.
To find out the exact location the following command can be used

$ strings $PWD/bin/myapp | egrep '.gcda$'
/home/bobah/Work/coverage/obj/main.gcda
/home/bobah/Work/coverage/obj/myclass.gcda

In many cases the coverage statistics should be collected from the application running on the environment (host, user etc.) other then the one where the application was built, so creating *.gcda files in the build directory may be impossible or impractical. To override the location to store the *.gcda files two environment variables can be used: GCOV_PREFIX and GCOV_PREFIX_STRIP. For example, if we want to replace the four leading elements in the path ("/home/bobah/Work/coverage") where myapp stores *.gcda with "/home/bobah/Work/cov_rpt" we would need to define the variables as follows (bash syntax) in the application's environment

export GCOV_PREFIX="/home/bobah/Work/cov_rpt"
export GCOV_PREFIX_STRIP=4

As a result of the override, the file myclass.gcda will be created in /home/bobah/Work/cov_rpt and not in /home/bobah/Work/coverage

Note, that for the post processing of the coverage data the most convenient way is to release source files and *.gcno artifacts to under $GCOV_PREFIX, so that a *.gcda file is created in the same directory as corresponding *.gcno
I do it using rsync like

rsync -acv --filter='+ */' --filter='+ *.cpp' --filter='+ *.h' --filter='+ *.gcno' --filter='- *' /home/bobah/Work/coverage/ /home/bobah/Work/cov_rpt

The coverage data is accumulated during subsequent application runs. To reset the counters either delete all *.gcda files under $GCOV_PREFIX directory

$ find $GCOV_PREFIX -type f -name '*.gcda' -print | xargs /bin/rm -f

Or use LCOV functionality

$ lcov --directory $GCOV_PREFIX --zerocounters

Post-processing runtime coverage statistics

I prefer using LCOV wrapper for the coverage data processing because it generates nicely looking HTML reports.
The essential bit here is that the source code tree is available in exactly the same place as it was during the build. This is required because gcov application behavior can't be manipulated using GCOV_PREFIX/GCOV_PREFIX_STRIP and it expects files exactly as they are stored in *.gcno files.
The data post processing:

$ lcov --directory $GCOV_PREFIX --capture --output-file $GCOV_PREFIX/app.info

The HTML reports generation:

$ genhtml --output-directory $PWD/cov_htmp $GCOV_PREFIX/app.info

Troubleshooting

"stamp mismatch with graph file" error message during gcov/lcov invocation

The *.gcno file contains a time stamp tag. The same tag is put to the runtime coverage report *.gcda file by the application. If the application and *.gcda files are created in the different build runs, gcov will refuse processing them.
The tag can be extracted from a file and compared *.gcda vs *.gcno

$ hexdump -e '"%x\n"' -s8 -n4 myclass.gcda
7ef26ee7
$ hexdump -e '"%x\n"' -s8 -n4 myclass.gcno
7ef26ee7

*.gcda files are not generated

Here there can be 2 possibilities, either the directory where the applkication wants to write *.gcda file does not exist or is not writable, or the application does not exit property by either returning from main() or calling exit(), typically when SIGTERM is not properly handled

Links

C++ Heap Map

A small malloc() and free() interceptor library (heaptrace.so, 45 lines of C++) and minimalistic TCL/TK GUI for visual C++ heap analysis — a powerful tool for disputes settlement and curiosity satisfaction.

An executable under investigation should run the following way (NEdit text editor in the example)

$ LD_PRELOAD=.../heaptrace.so .../nedit | awk '/^[+-]heap /' >.../heaptrace.log
An awk-based filter is suppressing the process's own stdout output.

Because of the preloaded interceptor library a message will be printed to the stdout on each malloc() or free() invocation. The resulting dump would look like

. . .
+heap 0x20b5040 64
+heap 0x20b5090 8176
+heap 0x20b7090 4096
+heap 0x20b80a0 32
+heap 0x20b80d0 8
+heap 0x20b80f0 568
-heap 0x20b80f0 0
+heap 0x20b80f0 2784
-heap 0x20b80f0 0
+heap 0x20b80f0 336
. . .

At the moment of interest the GUI script can be invoked on the resulting dump as

$ .../heapmap.tk .../nedit.allocs.txt
The GUI script will print some diagnostic information
reading nedit.allocs.txt
allocated memory blocks: 16278
input min address: 0x1dc1040
input max address: 0x2058aa0
aligned min address: 0x1dc1000
aligned max address: 0x2059000
page size: 0x1000 (4096)
number of pages: 0x298 (664)
max_pages_per_row=2.23606797749979
pages_per_row=2
allocations drawing done
vertical grid lines done
horizontal grid lines done
and will display the heap map heapmap.tk The map can be zoomed by GUI buttons or via left mouse button area selection and scrolled either with scroll bars or right mouse button drag. Double click on the allocated block highlights all other blocks of the same size on the map. heapmap.tk.zoom

C++ source code for the library (build instructions in the comments) is below. It is also in attachment to the post along with the GUI TCL/TK script (license)

#include <cstdio>
#include <dlfcn.h>

#define likely(x)   __builtin_expect((x),1)
#define unlikely(x) __builtin_expect((x),0)

// g++ -pthread -m64 -fPIC -std=c++0x -O3 -Wl,-zdefs -Wl,-znow -ldl -shared -Wl,-soname,heaptrace.so -o heaptrace.so heaptrace.cc

namespace
{

/**
 * malloc() direct call
 */
inline void * libc_malloc(size_t size)
{
  typedef void* (*malloc_func_t)(size_t);
  static malloc_func_t malloc_func = (malloc_func_t) dlsym(RTLD_NEXT, "malloc");

  return malloc_func(size);
}

/**
 * free() direct call
 */
inline void * libc_free(void* ptr)
{
  typedef void (*free_func_t)(void*);
  static free_func_t free_func = (free_func_t) dlsym(RTLD_NEXT, "free");

  free_func(ptr);
}

/**
 * malloc() call recorder
 */
void record_malloc(size_t size, void* ptr)
{
  if (unlikely(ptr == 0)) return;

  char buf[64];
  size_t len = snprintf(buf, sizeof(buf) / sizeof(char), "+heap %p %lu\n", ptr, size);
  fwrite(buf, sizeof(char), len, stdout);
}

/**
 * free() call recorder
 */
void record_free(void* ptr)
{
  if (unlikely(ptr == 0)) return;

  char buf[64];
  size_t len = snprintf(buf, sizeof(buf) / sizeof(char), "-heap %p 0\n", ptr);
  fwrite(buf, sizeof(char), len, stdout);
}

} // anonymous namespace


/**
 * malloc() override
 */
extern "C" void* malloc(size_t size)
{
  void* ptr = libc_malloc(size);
  record_malloc(size, ptr);
  return ptr;
}

/**
 * free() override
 */
extern "C" void free(void *ptr)
{
  libc_free(ptr);
  record_free(ptr);
  return;
}

On Linux where address randomization may take place one would need to run setarch -R in the shell to switch it of during the test.

AttachmentSize
heapmap.tk16.96 KB
heaptrace.cc1.45 KB

Executing Tasks in Parallel with Xargs

A neat single liner to run tasks in parallel with xargs command available on every Linux/Unix server.
#!/bin/bash

...
echo PARALLEL_JOBS:${PARALLEL_JOBS:=1}

declare -a tests=($(.../find_all_tests))
echo "${tests[@]}" | \
  xargs -d' ' -n1 -P${PARALLEL_JOBS} -I {} bash -c ".../run_test {}" || { echo "FAILURE"; exit 1; }

echo "SUCCESS"

Incremental Text File Monitoring

In many situations it is desired to monitor a log file for errors in incremental fashion, as if tail -f was run on it, but in batches, say from a cron-scheduled task. The subject script takes the filename as an input and outputs what changed from the last invocation (storing the state in the same directory as a subject file).
The script (whatsnew.awk)
#!/bin/awk -f
# Copyright (C) 2012 Vladimir Lysyy
# http://bobah.net/d4d/source-code/license

BEGIN {

    FS=" ";
    state_file="";
    last_line=0;
}

{
  if (NR == 1) {
    state_file = FILENAME ".lastline";
    sub(/^.*\//, "", state_file);
    getline last_line < state_file;
    print "state_file =", state_file, "last_line =", last_line
  }

  if (NR <= last_line) next;
}

{ print; }

END {
  print NR > state_file
}

Example invocation
$ whatsnew.awk /var/log/messages

Jackrabbit on Apache Tomcat

# download files
wget 'http://mirror.switch.ch/mirror/apache/dist/tomcat/tomcat-6/v6.0.26/bin/apache-tomcat-6.0.26.tar.gz'
wget 'http://www.apache.org/dyn/closer.cgi/jackrabbit/2.1.0/jackrabbit-webapp-2.1.0.war'
wget 'http://repo1.maven.org/maven2/javax/jcr/jcr/2.0/jcr-2.0.jar'

# extract tomcat
tar xzf apache-tomcat-6.0.26.tar.gz

# put JCR jar on classpath
cd apache-tomcat-6.0.26
mkdir -p ./shared/lib
cp ../jcr-2.0.jar ./shared/lib
sed -s 's@shared.loader=@shared.loader=${catalina.home}/shared/lib/jcr-2.0.jar,@' ./conf/catalina.properties

# start Tomcat
bin/startup.sh

# deploy Jackrabbit
cp ../jackrabbit-webapp-2.1.0.war ./webaps

Java Developer's Toolset

This small article is a reminder for myself, for the case when I need to set my workspace up next time. A set of short and most essential points which I don't want to ever re-discover again. I start from the description about the development, build, and testing environment and accepted practices and continue with the description (or just mentioning) of important software packages or frameworks (IO, logging, persistence, XML, etc.) which can really boost a developer's performance and are a must for your code to be "pluggable" like the rest of Java code.

A Convention

A basic things about Java development environments is a convention, as basic as a rule to put #ifndef...#define...#endif macros in C/C++ header files. Each class should belong to a package. A package name is reversed order corporate Internet domain name followed by dot separated hierarchical package name in direct order (like net.bobah.examples.logging). The package statement should be the first statement in the file. The files should be located in the directory tree according to their packages names. The file HelloWorld from net.bobah.examples.logging package should be put to .../net/bobah/examples/logging/HelloWorld.java). Once the convention is met the source tree can be used by most Java development frameworks and build systems without any refactoring.

IDE

I like the Eclipse platform (the one for Java EE), http://www.eclipse.org/downloads/moreinfo/jee.php. A full featured development framework written in Java and for Java. Very convenient for code browsing, refactoring, debugging. And building, but I personally prefer Ant (see next section "Building"). It also provides an excellent real-time compiler errors reporting. The most essential thing to know about Eclipse is a terminology:
project — a project, a set of files not having much sense without each other (usually implementing a well determined piece of functionality)
workspace — a set of projects having something in common (interdependencies, logical or runtime connection, common protocols, etc.), kind of "working context", the workspaces can be "switched" from one to another perspective — a configuration of a GUI's widgets layout and a functionality available via menus. There are, for instance "Java code browse" and "Java debug" perspectives. Perspectives themselves are pluggable, so one can have, say Hibernate perspective with special tools for persistence layer development.
The refactoring tools and automatic code generators for constructors and getters/setters are available via context menu (and shortcuts). "SHIFT+CTRL+T" is a sortcut for a class search within the workspace.
I'd recommend downloading and installing the Eclipse IDE for Java EE Developers as it much more feature reach than the one for Java SE (JSP/JSF, Web-Services, EJB, XML, etc.).

Building

The build is typically done either directly from Eclipse or using either Maven or Ant.

Eclipse

http://eclipse.org/. The simplest push-button build variant, good for the build debug or a prototype small project. The CLASSPATH, input and output directories are configuration is done via project properties menu. The project specific settings are stored in .project and .classpath files in the project's root directory. The workspace specific settings (code style, compiler settings, etc.) are configured via Window->Preferences, I never had to find either of these setting in the file system.

Ant

http://ant.apache.org/. Very similar to makefile but configuration files are in XML format and there is a set of Java specific atomic tasks so there is no need to memorize all javac flags and automate a filesystem iteration for compiling a source tree. My preferred build system. Scalable, extensible, configurable, yet simple and very fast.

Maven

http://maven.apache.org/. Convention enforcing, strict framework written in plugin-based architecture. Plugins exist for most common use cases. When using Maven the Eclipse project can be generated with this command: "mvn -DdownloadSources=true -DdownloadJavadocs=true -DoutputDirectory=target/eclipse-classes eclipse:clean eclipse:eclipse".

Already written

Testing - JUnit

http://www.junit.org/ a de facto standard for Java project unit testing.

Logging — SLF4J & Logback

http://slf4j.org// and http://logback.qos.ch/. The first is a logging abstraction layer, the letter is a feature rich, configurable, and reliable logger.

Persistence — Hibernate

https://www.hibernate.org/. JPA certified persistence framework. Very powerful and extremely easy to set up and use once you understand the concepts. Offers and excellent reverse engineering (DB-to-Java or DB-to-DDL) functionality. There is a Hibernate plugin for Eclipse offering visual configuration files creation with development time connection to the DB.
A good complimentary tool, a DBMS client written in Java is a SquirrelSQL, it let's you test a DB connection settings, SQL queries in a JDBC dialect and many more. And it is using the same code and drivers which your application would use.

Web Services & Web GUI (JSP) — Apache Tomcat, Servlet, JAX-WS, SoapUI

Lot of other stuff

There are lots of projects in the Apache Software Foundation which can be combined, reused (as a whole or partially) or copy-pasted from.

Simple Test Runner

300 lines of Bash code. Take and use.

Born on a rainy Herfordshire weekend, the Simple Test Runner, a balanced fuse of simplicity and power, is an essence of more than a decade of experience with software development processes automation at world's leading EDA companies and a top tier investment bank.

The Runner will be an ideal base for testing automation in any Bash-enabled Linux environment. The code base of just about 300 lines is meant to direct but not limit the user. A single file tests.txt is both a list of tests and a configuration.

Software Quality Assurance (QA)

Quality Assurance is the process that the testing is a part of. A test is a single QA task with well defined success criteria (e.g. software build is a QA step validating that the software builds, and is thus the first step in any QA process). Test is only worth conducting if it positively impacts the QA process (the cost to run the test is less than the penalty from the potential loss if not running it). Once automated, the test well amortizes across the life time of the software, providing that automation process maintenance is robust enough and is not an overhead on its own.

Tests are typically subdivided by categories (unit, functional, regression, performance, etc.), but it really does not matter for automation. What does matter is the cost of automation versus that of manual testing. In order for test to be reliably automated it should be possible to consistently isolate the entity being tested (for input and environment) and have a reliable success criteria capable of producing the binary yes/no answer for the test. Simple Test Runner is written to treat software QA as such.

Abstraction Levels

The Simple Test Runner has three levels of testing abstraction and includes just enough support for all of them. These levels are (from abstract to specific):

  1. Abstract Test - run test command and check its exit code
  2. Material Test - run test.sh script that takes three parameters, test setup command, test execution command, and test result analyzer command, all three must succeed (0 exit code) in order for test to be counted as passed
  3. Specific Test - run test.sh script same way as above but use analyzer.sh to analyze the output, the analyzer.sh expects that test command produces files and needs to be provided the directory with "golden" files to compare test output with
Start with the simplest one that solves your immediate task, check the next one when need more.
Distribution Archive Contents

suite
     \
      +-code     # Bash Code, ~319 lines
      +-example  # is worth a thousand words
      +-LICENSE  # to guard ourselves from each other

Minimum Setup

  1. Put the directory code somewhere and treat it read-only
  2. Create a file tests.txt in the directory where you run the test or, preferrably, create it somewhere else and create a symlink to it in the directory where you're going to run tests
If you change to the directory having tests.txt you should be able to run the script suite.sh. If you run it as suite.sh -n -g\* it will display all tests in the file tests.txt.

First Level - Abstract Test

Abstract test is only supposed to be a shell executable and thus be able to return exit status.

The file tests.txt can look as the one below:

# [group name] [test name]            [timeout, sec.] [command to run]
  run          trivial_pass_test      30              /bin/true
  run          trivial_fail_test      30              /bin/false
  run          trivial_timedout_test  3               /bin/sleep 5
The suite.sh runs test's command line and fetches the exit code if the code is 0 the test is PASSED, otherwise FAILED
bobah@europa> suite.sh -g run ;# -g: a group of tests to run, in this case "run"
suite.sh -I- run trivial_pass_test (pid=4813, pwd=run/trivial_pass_test/work)
suite.sh -I- trivial_pass_test - PASSED, 00:00:00 (0s)
suite.sh -I- run trivial_fail_test (pid=4821, pwd=run/trivial_fail_test/work)
suite.sh -I- trivial_fail_test - FAILED, rc=1, 00:00:01 (1s)
suite.sh -I- run trivial_timedout_test (pid=4831, pwd=run/trivial_timedout_test/work)
suite.sh -I- run trivial_timedout_test - TIMEOUT, terminated, 00:00:04 (4s)
suite.sh -I- total tests: 3, passed: 1, failed: 2

Second Level - Material Test

Material test is supposed to need pre-run preparation and produce analyzable results. For instance, preparation can be fetching the person's e-mail from address book, test execution can be sending a mail to the person and asking to reply, and test results analysis can be checking for the reply in the own mailbox.

In case your test fits in prepare-execute-analyze model you can use Simple Test Runner's test.sh script, which needs you to provide commands for three above described steps. Setup and Analyzer commands are defaulted to /bin/true, so if configured as below, it runs just as an Abstract Test, just testing the exit code of test executable.

# [group name] [test name]                [timeout, sec.] [command to run]
  run          abstract_as_material_pass  30              ${BASE_DIR}/test.sh -x /bin/true

The below tests.txt demonstrates all possibilities of the Material Test model implemented in test.sh

run  material_pass           30  ${BASE_DIR}/test.sh -x /bin/true
run  material_fail_setup     30  ${BASE_DIR}/test.sh -s /bin/false -x /bin/true  -a /bin/true
run  material_fail_running   30  ${BASE_DIR}/test.sh -s /bin/true  -x /bin/false -a /bin/true
run  material_fail_analyzis  30  ${BASE_DIR}/test.sh -s /bin/true  -x /bin/true  -a /bin/false

Third Level - Specific Test

Specific test is supposed to produce something in the directory where it runs. The canonical file-based test output analysis implementation is provided by the script analyzer.sh, which is also a part of the Simple Test Runner.

The analyzer.sh script expects three parameters: diff command (defaults to "diff -q"), filter command (defaults to /bin/cat), and a directory with golden output to compare the current output with. The work done by analyzer for each file in the output is schematically described as below.

current/outputfile.ext | filter | outputfile.txt.current \
                                                          --> diff_cmd ? PASS/FAIL
golden/outputfile.ext | filter | outputfile.txt.gold     /

AttachmentSize
str_v0.2.2.tgz5.08 KB

Source Code

Useful or just educational code which I've written not-for-sale. Published here in a hope that it can be useful and without any warranty, explicit or implicit.

Copyright & License


  Copyright 2001-2014 Vladimir Lysyy
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this source code except in compliance with the License.
  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.




Math

In this section I have collected a source code implementing various numeric methods and computational algorithms for problems from various applied fields. For most of them I intentionally don't give any supplementary info, only URL to the relevant Wikipedia and/or AlgoList articles.

Cubic Spline

A Java implementation of a cubic spline interpolation of a function defined by an array of points with fixed step. Theory here.

AttachmentSize
CubicSpline.zip2.19 KB

Networking

Networking code for copy/paste

SSL Client Socket

A minimal SSL client (also a TCP client). Demonstrates TCP and SSL client socket creation, initialization and usage by means of pure POSIX API.
 #include <openssl/ssl.h>
 #include <openssl/bio.h>
 #include <openssl/err.h>

 #include <cstdio>
 #include <cstring>

 #include <unistd.h>
 #include <fcntl.h>
 #include <sys/types.h>
 #include <errno.h>
 #include <assert.h>

 #include <sys/socket.h>
 #include <netinet/in.h>
 #include <netinet/tcp.h>
 #include <arpa/inet.h>
 #include <netdb.h>
 #include <sys/time.h>
 #include <string.h>

 #include <iostream>
 #include <iomanip>


 namespace {

 enum {
   FATAL = 0,
   ERROR,
   WARNING,
   INFO,
   DETAIL,
   DEBUG,
   TRACE
 };

 int _logLevel = DETAIL;

 #define LOG(level,msg) do { if (level <= _logLevel) {std::cout << msg; } } while(0)
 #define LOG_ERROR(msg) do { LOG(ERROR, "-E- " << __func__ << ' ' << msg << std::endl); } while(0)
 #define LOG_DEBUG(msg) do { LOG(DEBUG, "-D- " << __func__ << ' ' << msg << '\n'); } while(0)
 #define LOG_INFO(msg)  do { LOG(INFO,  "-I- " << __func__ << ' ' << msg << '\n'); } while(0)

 #define PERROR_AND_RETURN(rc) do {                                           \
   LOG_ERROR(strerror(errno) << '(' << errno << ')'); \
   return rc;                                                                        \
 } while (0)


 struct addrinfo* get_addrinfo(const char* node, const char* service, bool tcp) {
   struct addrinfo  hints  = addrinfo();
   struct addrinfo* result = NULL;

   memset(&hints, 0, sizeof(struct addrinfo));
   hints.ai_family    = AF_UNSPEC;    /* Allow IPv4 or IPv6 */
   hints.ai_socktype  = tcp ? SOCK_STREAM : SOCK_DGRAM; /* Stream/Datagram socket */
   hints.ai_flags     = 0;
   hints.ai_protocol  = 0;          /* Any protocol */

   int rc = 0;
   if (getaddrinfo(node, service, &hints, &result) != 0) { LOG_ERROR(gai_strerror(rc)); return NULL; }

   return result;
 }

 /**
  * 
  */
 int configure_socket(int fd, const struct addrinfo& ai, unsigned sndbuf, unsigned rcvbuf, bool blocking, bool nolinger, bool nodelay) {
   if (nolinger) {
     /* Set the socket for a non lingering, graceful close.
      * This will cause a final close of this socket not to wait until all
      * of the data sent on it has been received by the remote host.
      * The result is that the socket will be immediately released instead
      * of blocking in a TIME_WAIT state */
     linger linger = {1, 0};
     if (::setsockopt(fd, SOL_SOCKET, SO_LINGER, &linger, sizeof(linger)) == -1) { LOG_ERROR(strerror(errno)); return -1; }
   }

   if (nodelay) {
     int flag = 1;
     if (::setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, &flag, sizeof(flag)) == -1) { LOG_ERROR(strerror(errno)); return -1; }
   }

   if (sndbuf > 0 && setsockopt(fd, SOL_SOCKET, SO_SNDBUF, &sndbuf, sizeof(sndbuf)) == -1) { LOG_ERROR(strerror(errno)); return -1; }
   if (rcvbuf > 0 && setsockopt(fd, SOL_SOCKET, SO_RCVBUF, &rcvbuf, sizeof(rcvbuf)) == -1) { LOG_ERROR(strerror(errno)); return -1; }

   // put socket to blocking/non-blocking mode
   int flags = -1;
   if ((flags = fcntl(fd, F_GETFL, 0)) == -1) { LOG_ERROR(strerror(errno)); return -1; }
   if (blocking) flags &= ~O_NONBLOCK;
   else          flags |= O_NONBLOCK;
   if (fcntl(fd, F_SETFL, flags) == -1) { LOG_ERROR(strerror(errno)); return -1; }

   return 0;
 }


 /**
  * returns tcp-connected socket
  */
 int get_tcp_connection(const char* node, const char* service) {
   struct addrinfo* ai = get_addrinfo(node, service, true /* tcp */);
   if (ai == NULL) return -1;

   const unsigned sndbuf = 1024*128;
   const unsigned rcvbuf = 1024*128;

   int fd = -1;
   for (struct addrinfo* rp = ai; rp != NULL; rp = rp->ai_next) {
     if ((fd = socket(rp->ai_family, rp->ai_socktype, rp->ai_protocol)) == -1) { LOG_ERROR(strerror(errno)); fd = -1; break; }
     if (configure_socket(fd, *rp, sndbuf, rcvbuf, true /* blocking */, true /* nolinger */, true /* nodelay */) != 0) { ::close(fd); fd = -1; break; }

     if (::connect(fd, rp->ai_addr, rp->ai_addrlen) != -1) break;
     ::close(fd); fd = -1;
   }

   ::freeaddrinfo(ai);
   return fd;
 }

 }

 int main()
 {
     int p;

     const char* request = "GET /\n\n";
     char r[1024];

     /* Set up the library */
     SSL_library_init();
     ERR_load_BIO_strings();
     SSL_load_error_strings();
     OpenSSL_add_all_algorithms();

     /* Set up the SSL context */

     SSL_CTX* ctx = SSL_CTX_new(SSLv3_method());
     if (!ctx) {
       ERR_print_errors_fp(stderr);
       return 1;
     }

     /* Load the trust store */

     if(! SSL_CTX_load_verify_locations(ctx, NULL, "/etc/ssl/certs"))
     {
         fprintf(stderr, "Error loading trust store\n");
         ERR_print_errors_fp(stderr);
         SSL_CTX_free(ctx);
         return 0;
     }

     /* Setup the connection */

     int fd = get_tcp_connection("127.0.0.1", "12345");
 //    int fd = my_connect("127.0.0.1", 80);
     if (fd == -1) {
       LOG_ERROR(strerror(errno));
       return 1;
     }


     SSL* ssl = SSL_new(ctx); assert(ssl);
     if (!SSL_set_fd(ssl, fd)) {
       LOG_ERROR("SSL_set_fd(ssl, fd)");
       return 1;
     }

     SSL_set_mode(ssl, SSL_MODE_AUTO_RETRY);

     int rc = SSL_get_error(ssl, SSL_connect(ssl));
     if (rc != SSL_ERROR_NONE) {
       LOG_ERROR(ERR_error_string(rc, NULL));
       while ((rc = ERR_get_error()) != 0) {
         LOG_ERROR(ERR_error_string(rc, NULL));
       }
       return 1;
     }

     /* Check the certificate */

     rc = SSL_get_verify_result(ssl);
     if(rc != X509_V_OK)
     {
         if (rc == X509_V_ERR_DEPTH_ZERO_SELF_SIGNED_CERT || rc == X509_V_ERR_SELF_SIGNED_CERT_IN_CHAIN) {
           fprintf(stderr, "self signed certificate\n");
         }
         else {
           fprintf(stderr, "Certificate verification error: %ld\n", SSL_get_verify_result(ssl));
           SSL_CTX_free(ctx);
           return 0;
         }
     }

     /* Send the request */
     if (SSL_write(ssl, request, strlen(request)) == -1) {
       PERROR_AND_RETURN(1);
     }
     LOG_INFO("Data sent [" << request << "]");

     /* Read in the response */

     for(;;)
     {
         p = SSL_read(ssl, r, 1023);
         if(p <= 0) break;
         r[p] = 0;
         printf("%s", r);
     }


     rc = SSL_shutdown(ssl);
     if(!rc){
       /* If we called SSL_shutdown() first then
          we always get return value of '0'. In
          this case, try again, but first send a
          TCP FIN to trigger the other side's
          close_notify*/
       shutdown(fd,1);
       rc = SSL_shutdown(ssl);
     }
     switch(rc){
       case 1:
         break; /* Success */
       case 0:
       case -1:
       default:
         LOG_ERROR("Shutdown failed");
     }

     /* Close the connection and free the context */

     SSL_CTX_free(ctx);
 //     if (write(fd, request, strlen(request)) == -1) {
 //       PERROR_AND_RETURN("write(fd, request, strlen(request))", 1);
 //     }
 // 
 //     /* Read in the response */
 // 
 //     for(;;)
 //     {
 //         p = read(fd, r, 1023);
 //         if(p <= 0) break;
 //         r[p] = 0;
 //         printf("%s", r);
 //     }
     return 0;
 }

SSL Server Socket

A minimum TCP/SSL server, demonstrates the server socket creation, initialization, and usage, as well as and server side SSL handshake part.
 #include <openssl/ssl.h>
 #include <openssl/bio.h>
 #include <openssl/err.h>

 #include <errno.h>
 #include <assert.h>

 #include <sys/types.h>
 #include <cstdio>
 #include <cstdlib>
 #include <unistd.h>
 #include <cstring>
 #include <sys/socket.h>
 #include <netdb.h>

 #define BUF_SIZE 500

 #define LISTEN_BACKLOG 50

 #include 
 #include 

 namespace {

 enum {
   FATAL = 0,
   ERROR,
   WARNING,
   INFO,
   DETAIL,
   DEBUG,
   TRACE
 };

 int _logLevel = DETAIL;

 #define LOG(level,msg) do { if (level <= _logLevel) {std::cout << msg; } } while(0)
 #define LOG_ERROR(msg) do { LOG(ERROR, "-E- " << __func__ << ' ' << msg << std::endl); } while(0)
 #define LOG_DEBUG(msg) do { LOG(DEBUG, "-D- " << __func__ << ' ' << msg << '\n'); } while(0)
 #define LOG_INFO(msg)  do { LOG(INFO,  "-I- " << __func__ << ' ' << msg << '\n'); } while(0)

 #define PERROR_AND_RETURN(rc) do {                                           \
   LOG_ERROR(strerror(errno) << '(' << errno << ')'); \
   return rc;                                                                        \
 } while (0)
 }

 int
 main(int argc, char *argv[])
 {
    struct addrinfo hints;
    struct addrinfo *result, *rp;
    int sfd, s;
    struct sockaddr_storage peer_addr;
    socklen_t peer_addr_len;
    ssize_t nread;
    char buf[BUF_SIZE];

    if (argc != 2) {
        fprintf(stderr, "Usage: %s port\n", argv[0]);
        exit(EXIT_FAILURE);
    }

     /* Set up the library */
     SSL_library_init();
     ERR_load_BIO_strings();
     SSL_load_error_strings();
     OpenSSL_add_all_algorithms();

     /* Set up the SSL context */

     SSL_CTX* ctx = SSL_CTX_new(SSLv3_method());
     if (!ctx) {
       ERR_print_errors_fp(stderr);
       return 1;
     }

     /* Load the trust store */

     if(! SSL_CTX_load_verify_locations(ctx, NULL, "/etc/ssl/certs"))
     {
         fprintf(stderr, "Error loading trust store\n");
         ERR_print_errors_fp(stderr);
         SSL_CTX_free(ctx);
         return 0;
     }

     if (!SSL_CTX_use_PrivateKey_file(ctx, "cert/server.key", SSL_FILETYPE_PEM)) {
       int rc;
       while ((rc = ERR_get_error()) != 0) {
         LOG_ERROR(ERR_error_string(rc, NULL));
       }
       return 1; // TODO: add SSL_ERROR_WANT_CONNECT loop around
     }
     LOG_INFO("server key ready");

     if (!SSL_CTX_use_certificate_file(ctx, "cert/server.crt", SSL_FILETYPE_PEM)) {
       int rc;
       while ((rc = ERR_get_error()) != 0) {
         LOG_ERROR(ERR_error_string(rc, NULL));
       }
       return 1; // TODO: add SSL_ERROR_WANT_CONNECT loop around
     }
     LOG_INFO("server cert ready");


    memset(&hints, 0, sizeof(struct addrinfo));
    hints.ai_family = AF_UNSPEC;    /* Allow IPv4 or IPv6 */
    hints.ai_socktype = SOCK_STREAM; /* Datagram socket */
    hints.ai_flags = AI_PASSIVE;    /* For wildcard IP address */
    hints.ai_protocol = 0;          /* Any protocol */
    hints.ai_canonname = NULL;
    hints.ai_addr = NULL;
    hints.ai_next = NULL;

    s = getaddrinfo(NULL, argv[1], &hints, &result);
    if (s != 0) {
        fprintf(stderr, "getaddrinfo: %s\n", gai_strerror(s));
        exit(EXIT_FAILURE);
    }

    /* getaddrinfo() returns a list of address structures.
       Try each address until we successfully bind(2).
       If socket(2) (or bind(2)) fails, we (close the socket
       and) try the next address. */

    for (rp = result; rp != NULL; rp = rp->ai_next) {
        sfd = socket(rp->ai_family, rp->ai_socktype,
                rp->ai_protocol);
        if (sfd == -1)
            continue;

        if (bind(sfd, rp->ai_addr, rp->ai_addrlen) == 0)
            break;                  /* Success */

        close(sfd);
    }

    if (rp == NULL) {               /* No address succeeded */
        fprintf(stderr, "Could not bind\n");
        exit(EXIT_FAILURE);
    }

    freeaddrinfo(result);           /* No longer needed */

    if (listen(sfd, LISTEN_BACKLOG) == -1) {
      fprintf(stderr, "%s\n", strerror(errno));
      exit(EXIT_FAILURE);
    }

    peer_addr_len = sizeof(struct sockaddr_storage);
    int cfd = accept(sfd, (struct sockaddr *) &peer_addr, &peer_addr_len);
    if (cfd == -1) {
      fprintf(stderr, "%s\n", strerror(errno));
      exit(EXIT_FAILURE);
    }

    char hbuf[NI_MAXHOST], sbuf[NI_MAXSERV];
    s = getnameinfo((struct sockaddr *) &peer_addr, peer_addr_len, hbuf, NI_MAXHOST, sbuf, NI_MAXSERV, NI_NUMERICSERV);
    if (s == 0) printf("Connection from %s:%s\n", hbuf, sbuf);
    else        fprintf(stderr, "getnameinfo: %s\n", gai_strerror(s));

    // **
    // * TCP socket ready
    // **

     SSL* ssl = SSL_new(ctx); assert(ssl);
     if (!SSL_set_fd(ssl, cfd)) {
       LOG_ERROR("SSL_set_fd(ssl, fd)");
       return 1;
     }

     SSL_set_mode(ssl, SSL_MODE_AUTO_RETRY);

     int rc = SSL_get_error(ssl, SSL_accept(ssl)); // TODO: is it thread-safe?
     if (rc != SSL_ERROR_NONE) {
       LOG_ERROR(ERR_error_string(rc, NULL));
       while ((rc = ERR_get_error()) != 0) {
         LOG_ERROR(ERR_error_string(rc, NULL));
       }
       return 1; // TODO: add SSL_ERROR_WANT_CONNECT loop around
     }

     rc = SSL_read(ssl, buf, BUF_SIZE);
     if (rc > 0)  {
       printf("Received %ld bytes from %s:%s\n", (long) rc, hbuf, sbuf);
     }

     const char* msg = "HELLO CLIENT, I AM SERVER\n";
     rc = SSL_write(ssl, msg, strlen(msg));
     if (rc > 0)  {
       printf("Received %ld bytes from %s:%s\n", (long) rc, hbuf, sbuf);
     }

     rc = SSL_shutdown(ssl);
     if(!rc){
       /* If we called SSL_shutdown() first then
          we always get return value of '0'. In
          this case, try again, but first send a
          TCP FIN to trigger the other side's
          close_notify*/
       shutdown(cfd,1);
       rc = SSL_shutdown(ssl);
     }
     switch(rc){
       case 1:
         break; /* Success */
       case 0:
       case -1:
       default:
         LOG_ERROR("Shutdown failed");
     }

     /* Close the connection and free the context */

     SSL_CTX_free(ctx);
   return 0;
 }

TCP Client (Java NIO)

A minimalistic Java NIO TCP client. Stays always connected. Disconnects and reconnects on any exception in the event handlers (onConnect, onDisconnect, onRead).
package net.bobah.nio;

import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.SocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.ReadableByteChannel;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;
import java.nio.channels.WritableByteChannel;
import java.util.Iterator;
import java.util.Random;
import java.util.Set;
import java.util.Timer;
import java.util.TimerTask;
import java.util.concurrent.PriorityBlockingQueue;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicLong;

import javax.annotation.PostConstruct;

import org.apache.log4j.BasicConfigurator;
import org.apache.log4j.ConsoleAppender;
import org.apache.log4j.Level;
import org.apache.log4j.Logger;
import org.apache.log4j.PatternLayout;

/**
 * A simple NIO TCP client
 * Assumptions:
 * - the client should always be connected,
 *   once it gets disconnected it reconnects
 * - the exception thrown by onRead means protocol error
 *   so client disconnects and reconnects
 * - the incoming flow is higher than outgoing, so
 *   direct channel write method is not implemented
 * 
 * @author Vladimir Lysyy (mail@bobah.net)
 *
 */
public abstract class TcpClient implements Runnable {
  protected static final Logger LOG = Logger.getLogger(TcpClient.class);
  private static final long INITIAL_RECONNECT_INTERVAL = 500; // 500 ms.
  private static final long MAXIMUM_RECONNECT_INTERVAL = 30000; // 30 sec.
  private static final int READ_BUFFER_SIZE = 0x100000;
  private static final int WRITE_BUFFER_SIZE = 0x100000;

  private long reconnectInterval = INITIAL_RECONNECT_INTERVAL;

  private ByteBuffer readBuf = ByteBuffer.allocateDirect(READ_BUFFER_SIZE); // 1Mb
  private ByteBuffer writeBuf = ByteBuffer.allocateDirect(WRITE_BUFFER_SIZE); // 1Mb

  private final Thread thread = new Thread(this);
  private SocketAddress address;

  private Selector selector;
  private SocketChannel channel;

  private final AtomicBoolean connected = new AtomicBoolean(false);

  private AtomicLong bytesOut = new AtomicLong(0L);
  private AtomicLong bytesIn = new AtomicLong(0L);

  public TcpClient() {
    
  }

  @PostConstruct
  public void init() {
    assert address != null: "server address missing";
  }

  public void start() throws IOException {
    LOG.info("starting event loop");
    thread.start();
  }

  public void join() throws InterruptedException {
    if (Thread.currentThread().getId() != thread.getId()) thread.join();
  }

  public void stop() throws IOException, InterruptedException {
    LOG.info("stopping event loop");
    thread.interrupt();
    selector.wakeup();
  }

  public boolean isConnected() {
    return connected.get();
  }

  /**
   * @param buffer data to send, the buffer should be flipped (ready for read)
   * @throws InterruptedException
   * @throws IOException
   */
  public void send(ByteBuffer buffer) throws InterruptedException, IOException {
    if (!connected.get()) throw new IOException("not connected");
    synchronized (writeBuf) {
      // try direct write of what's in the buffer to free up space
      if (writeBuf.remaining() < buffer.remaining()) {
        writeBuf.flip();
        int bytesOp = 0, bytesTotal = 0;
        while (writeBuf.hasRemaining() && (bytesOp = channel.write(writeBuf)) > 0) bytesTotal += bytesOp;
        writeBuf.compact();
      }

      // if didn't help, wait till some space appears
      if (Thread.currentThread().getId() != thread.getId()) {
        while (writeBuf.remaining() < buffer.remaining()) writeBuf.wait();
      }
      else {
        if (writeBuf.remaining() < buffer.remaining()) throw new IOException("send buffer full"); // TODO: add reallocation or buffers chain
      }
      writeBuf.put(buffer);

      // try direct write to decrease the latency
      writeBuf.flip();
      int bytesOp = 0, bytesTotal = 0;
      while (writeBuf.hasRemaining() && (bytesOp = channel.write(writeBuf)) > 0) bytesTotal += bytesOp;
      writeBuf.compact();

      if (writeBuf.hasRemaining()) {
        SelectionKey key = channel.keyFor(selector);
        key.interestOps(key.interestOps() | SelectionKey.OP_WRITE);
        selector.wakeup();
      }
    }
  }

  /**
   * Override with something meaningful
   * @param buf
   */
  protected abstract void onRead(ByteBuffer buf) throws Exception;
  
  /**
   * Override with something meaningful
   * @param buf
   */
  protected abstract void onConnected() throws Exception;

  /**
   * Override with something meaningful
   * @param buf
   */
  protected abstract void onDisconnected();

  private void configureChannel(SocketChannel channel) throws IOException {
    channel.configureBlocking(false);
    channel.socket().setSendBufferSize(0x100000); // 1Mb
    channel.socket().setReceiveBufferSize(0x100000); // 1Mb
    channel.socket().setKeepAlive(true);
    channel.socket().setReuseAddress(true);
    channel.socket().setSoLinger(false, 0);
    channel.socket().setSoTimeout(0);
    channel.socket().setTcpNoDelay(true);
  }

  @Override
  public void run() {
    LOG.info("event loop running");
    try {
      while(! Thread.interrupted()) { // reconnection loop
        try {
          selector = Selector.open();
          channel = SocketChannel.open();
          configureChannel(channel);

          channel.connect(address);
          channel.register(selector, SelectionKey.OP_CONNECT);

          while(!thread.isInterrupted() && channel.isOpen()) { // events multiplexing loop
            if (selector.select() > 0) processSelectedKeys(selector.selectedKeys());
          }
        } catch (Exception e) {
          LOG.error("exception", e);
        } finally {
          connected.set(false);
          onDisconnected();
          writeBuf.clear();
          readBuf.clear();
          if (channel != null) channel.close();
          if (selector != null) selector.close();
          LOG.info("connection closed");
        }

        try {
          Thread.sleep(reconnectInterval);
          if (reconnectInterval < MAXIMUM_RECONNECT_INTERVAL) reconnectInterval *= 2;
          LOG.info("reconnecting to " + address);
        } catch (InterruptedException e) {
          break;
        }
      }
    } catch (Exception e) {
      LOG.error("unrecoverable error", e);
    }
   
    LOG.info("event loop terminated");
  }

  private void processSelectedKeys(Set keys) throws Exception {
    Iterator itr = keys.iterator();
    while (itr.hasNext()) {
      SelectionKey key = itr.next();
      if (key.isReadable()) processRead(key);
      if (key.isWritable()) processWrite(key);
      if (key.isConnectable()) processConnect(key);
      if (key.isAcceptable()) ;
      itr.remove();
    }
  }

  private void processConnect(SelectionKey key) throws Exception {
    SocketChannel ch = (SocketChannel) key.channel();
    if (ch.finishConnect()) {
      LOG.info("connected to " + address);
      key.interestOps(key.interestOps() ^ SelectionKey.OP_CONNECT);
      key.interestOps(key.interestOps() | SelectionKey.OP_READ);
      reconnectInterval = INITIAL_RECONNECT_INTERVAL;
      connected.set(true);
      onConnected();
    }
  }

  private void processRead(SelectionKey key) throws Exception {
    ReadableByteChannel ch = (ReadableByteChannel)key.channel();

    int bytesOp = 0, bytesTotal = 0;
    while (readBuf.hasRemaining() && (bytesOp = ch.read(readBuf)) > 0) bytesTotal += bytesOp;

    if (bytesTotal > 0) {
      readBuf.flip();
      onRead(readBuf);
      readBuf.compact();
    }
    else if (bytesOp == -1) {
      LOG.info("peer closed read channel");
      ch.close();
    }

    bytesIn.addAndGet(bytesTotal);
  }

  private void processWrite(SelectionKey key) throws IOException {
    WritableByteChannel ch = (WritableByteChannel)key.channel();
    synchronized (writeBuf) {
      writeBuf.flip();

      int bytesOp = 0, bytesTotal = 0;
      while (writeBuf.hasRemaining() && (bytesOp = ch.write(writeBuf)) > 0) bytesTotal += bytesOp;

      bytesOut.addAndGet(bytesTotal);

      if (writeBuf.remaining() == 0) {
        key.interestOps(key.interestOps() ^ SelectionKey.OP_WRITE);
      }

      if (bytesTotal > 0) writeBuf.notify();
      else if (bytesOp == -1) {
        LOG.info("peer closed write channel");
        ch.close();
      }

      writeBuf.compact();
    }
  }

  public SocketAddress getAddress() {
    return address;
  }

  public void setAddress(SocketAddress address) {
    this.address = address;
  }

  public long getBytesOut() {
    return bytesOut.get();
  }

  public long getBytesIn() {
    return bytesIn.get();
  }


  /**
   * can be used for testing
   */
  public static void main(String[] args) throws Exception {
    BasicConfigurator.configure(new ConsoleAppender(new PatternLayout("%d{yyyyMMdd-HH:mm:ss} %-10t %-5p %-20C{1} - %m%n")));
    Logger.getRootLogger().setLevel(Level.INFO);
    final TcpClient client = new TcpClient() {
      @Override protected void onRead(ByteBuffer buf) throws Exception { buf.position(buf.limit()); }
      @Override protected void onDisconnected() { }
      @Override protected void onConnected() throws Exception { }
    };

    client.setAddress(new InetSocketAddress("127.0.0.1", 20001));
    try {
      client.start();
    } catch (IOException e) {
      e.printStackTrace();
    }

    Timer timer = new Timer();
    timer.schedule(new TimerTask() {
      @Override
      public void run() {
        LOG.info("out bytes: " + client.bytesOut.get());
        LOG.info("in bytes:  " + client.bytesIn.get());
      }
    }, 5000, 5000);

    while(!client.isConnected()) Thread.sleep(500);

    LOG.info("starting server flood");
    ByteBuffer buf = ByteBuffer.allocate(65535);
    Random rnd = new Random();
    while (true) {
      short len = (short) rnd.nextInt(Short.MAX_VALUE - 2);
      byte[] bytes = new byte[len];
      rnd.nextBytes(bytes);
      buf.putShort((short)len);
      buf.put(bytes);
      buf.flip();
      try {
        client.send(buf);
      } catch (Exception e) {
        LOG.error("exception: " + e.getMessage());
        while (!client.isConnected()) Thread.sleep(1000);
      }
      buf.clear();
      Thread.sleep(10);
    }
  }
}

Miscelanea

Pieces of code which can find their place in any kind of software

Java Glob Pattern Matcher

The below code is a simple glob-style (asterisk) pattern matcher. For those who doesn't want to use relatively slow regexp-based emulation or third-party libraries. The code is ~20 lines only (the remaining part, main(), is just for benchmarking. Copy/paste and enjoy.
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

/**
 *  The glob pattern matching supporting only asterisk wildcard '*'
 * is enough for many applications and can be implemented with several
 * lines of code. It is much faster then regexp-based emulation.
 *
 * @author Vladimir Lysyy
 */
public class PatternTest {
  /**
   * Matches line against pattern
   * @param pattern, the array of pattern substrings divided by asterisks
   * @param line
   * @return true if line matches pattern and false otherwise
   */
  public static boolean miniglob(String[] pattern, String line) {
    if (pattern.length == 0) return line.isEmpty();
    else if (pattern.length == 1) return line.equals(pattern[0]);
    else {
      if (!line.startsWith(pattern[0])) return false;
      int idx = pattern[0].length();
      for (int i = 1; i < pattern.length - 1; ++i) {
        String patternTok = pattern[i];
        int nextIdx = line.indexOf(patternTok, idx);
        if (nextIdx < 0) return false;
        else idx = nextIdx + patternTok.length();
      }
      if (!line.endsWith(pattern[pattern.length - 1])) return false;
      return true;
    }
  }

  /**
   * @param args
   */
  public static void main(String[] args) {
    BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
    try {
      // read from stdin space separated text and pattern
      for (String input = in.readLine(); input != null; input = in.readLine()) {
        String[] tokens = input.split(" ");
        String line = tokens[0];
        String[] pattern = tokens[1].split("\\*+", -1 /* want empty trailing token if any */);
        
        // check matcher performance
        long tm0 = System.currentTimeMillis();
        for (int i = 0; i < 1000000; ++i) {
          miniglob(pattern, line);
        }
        long tm1 = System.currentTimeMillis();
        System.out.println("miniglob took " + (tm1-tm0) + " ms");
        
        // check regexp performance
        Pattern reptn = Pattern.compile(tokens[1].replace("*", ".*"));
        Matcher mtchr = reptn.matcher(line);
        tm0 = System.currentTimeMillis();
        for (int i = 0; i < 1000000; ++i) {
          mtchr.matches();
        }
        tm1 = System.currentTimeMillis();
        System.out.println("regexp took " + (tm1-tm0) + " ms");

        // check if miniglob worked correctly
        if (miniglob(pattern, line)) {
          System.out.println("+ >" + line);
        }
        else {
          System.out.println("- >" + line);
        }
      }
    } catch (IOException e) {
      // TODO Auto-generated catch block
      e.printStackTrace();
    }
  }
}

Log4j Configuration and Usage

Log4j is a powerful and easy to configure logger from Apache Foundation. For the detailed instructions and to download the logger visit the log4j homepage. In this article I've published a minimum set of the code for copy/paste required to enable logging with log4j in an application. In the static initialization section (e.g. in main()) should be something like
  assert(System.getProperty("log4j.configuration") != null);
  org.apache.log4j.xml.DOMConfigurator.configure(System.getProperty("log4j.configuration")); 
or, if you don't have a plan of making the logger configurable
  System.setProperty("log4j.defaultInitOverride","true");
  BasicConfigurator.configure(new ConsoleAppender(new SimpleLayout());
or just
  BasicConfigurator.configure();
The logger-enabled class can look somewhat like this
package net.bobah.samples.log4j;
public class ClassWithLogging {
  private static final org.apache.log4j.Logger LOG = org.apache.log4j.Logger.getLogger(ClassWithLogging.class);

  public void logSomething() {
    LOG.fatal("fatal error sample", new RuntimeException("fake runtime exception"));
    LOG.error("error message sample", new IOException("fake IO exception"));
    LOG.warn("warning message sample");
    LOG.info("info message sample");
    // debug message, minimizing overhead in non-debug mode
    if (LOG.isDebugEnabled()) LOG.debug("Some" + " long" + " debug" + " message");
    if (LOG.isTraceEnabled()) LOG.trace("Some" + " long" + " trace" + " message");
  }

  public static void main() {
    assert(System.getProperty("log4j.config") != null);
    org.apache.log4j.xml.DOMConfigurator.configure(System.getProperty("log4j.config"));
    new ClassWithLogging().logSomething();
  }
}
The application should be started with -Dlog4j.config= or the file log4j.xml should be available on the CLASSPATH The config file can be like the one below, which gives a meaningful message prefix and logfile daily rolling feature. A special care should be taken when composing a definition for a pattern layout as components like %C make logger really slow (everything that would print the runtime invocation point information should be avoided, as log4j determines the invocation point by throwing and catching a dummy exception for this purpose).
<?xml version="1.0" encoding="UTF-8" ?>




  
    
    
      
    
  

  
      
      
      
          
      
  

  
    
    
  


or the similar using property configurator
log4j.rootLogger=INFO, file
log4j.throwableRenderer=org.apache.log4j.EnhancedThrowableRenderer

log4j.appender.file=org.apache.log4j.DailyRollingFileAppender
log4j.appender.file.File=${ENV_LOG_FILE}
log4j.appender.file.Append=true
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d{yyyyMMdd-HH:mm:ss.SSS} %t %p %c{1} - %m%n

log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{yyyyMMdd-HH:mm:ss.SSS} %t %p %c{1} - %m%n

log4j.logger.net.bobah=DEBUG

Context Switch Benchmark (POSIX, C++)

Sample code for context switching speed benchmarking on POSIX (pthreads, tested on Linux). The code also demonstrates how alarm signal can be used for printing regular stats.
#include <sched.h>
#include <sys/time.h>
#include <signal.h>
#include <pthread.h>

#include <cstdint>
#include <cstdlib>

#include <iostream>


#define NUM_THREADS 20


namespace {

pthread_mutex_t mtx = PTHREAD_MUTEX_INITIALIZER;

volatile long switches = 0L;
long msec = 0L;

/**
 * Current GMT time in ms since the Epoch
 */
uint64_t epochmsec() {
 timeval tv = timeval();
 if (gettimeofday(&tv, 0) == 0) {
   return tv.tv_sec * 1000 + tv.tv_usec / 1000;
 }
 else {
   return 0L;
 }
}

/**
 * ALRM signal handler, logs context switching statistics
 */
void alarm_handler(int signum) {
 alarm(1);
 uint64_t tm = epochmsec();
 uint64_t delta = tm - msec;
 msec = tm;
 std::cout << (delta / (double) switches * 1000) << "us" << std::endl;
 switches = 0;
}

/**
 * Thread function
 */
void* thread_func(void* arg) {
 for(;;) {
   pthread_mutex_lock(&mtx);
   ++switches;
   pthread_mutex_unlock(&mtx);
   sched_yield();
 }
}

} // namespace

int main() {
 msec = epochmsec();

 pthread_t threads[NUM_THREADS];

 for (size_t i = 0; i < sizeof(threads) / sizeof(pthread_t); ++i) {
   pthread_create(&threads[i], 0, &thread_func, 0);
 }

 // setup an alarm to print log message every second
 signal(SIGALRM, &alarm_handler);
 alarm(1);

 for (;;) {
   sleep(1000);
 }

 return 0;
}

Advanced EasyMock Usecases

void method testing
    mockObject.voidMethod(voidMethodArg);
    EasyMock.expectLastCall().once(); // or any other expectations about above call

C++ functor with signature of a class method


template <class F> struct signature_helper;
template <class T, class R, class... Args> struct signature_helper<R(T::*)(Args...)> { using type = std::function<R(Args...)>; };
template <class T, class R, class... Args> struct signature_helper<R(T::*)(Args...)const>: signature_helper<R(T::*)(Args...)> {};

Dependency Injection in C++

Dependency Injection is a powerful software design technique promoting component-based design and enabling complete component logic isolation from initialization time boilerplate. As a useful consequence, components written for dependency injection have well defined interfaces, can be exhaustively tested with unit tests and can thus be used as generic architectural building block without the fear of exciting a new execution path (opposed to typically solid design of software systems written in a chaotic, "Jira-by-Jira" way). Java community can use dependency injection with minimum effort when using Spring Framework, whereas C++ community is traditionally ignorant to the maintenance and code reuse benefits so there is no single leader around. The article is a step-by-step how-to instruction on enabling dependency injection in C++ code on Linux.
Separate Bootstrapping and Business Logic
WIP
Separate Interface and Implementation
WIP
Use Dynamic Linking
WIP
Enforce Build Time Unresolved Symbols Check
The last, but not the least — build time hygiene. The Makefile used in the project is below. It demonstrates how to correctly build dynamic libraries on GNU Linux to make sure no runtime surprises with unresolved symbols ever occur.
CXX=g++
CPPFLAGS:=-MMD -I/home/bobah/work/cpplibs/boost_1_47_0
CXXFLAGS:=-m64 -fPIC
LDFLAGS:=-m64 -fPIC -Wl,-zdefs -Wl,-znow
LOADLIBES:=
LDLIBS:=

.PHONY: all
all: app libclient.so libmodule.so

.PHONY: clean
clean:
	/bin/rm -f app libclient.so libmodule.so

libmodule.so: module.cc
	$(CXX) -shared -Wl,-soname,$@.1 -o $@ $^ -m64 -fPIC -Wl,-zdefs -Wl,-znow 

libclient.so: client.cc
	$(CXX) -shared -Wl,-soname,$@.1 -o $@ $^ -m64 -fPIC -Wl,-zdefs -Wl,-znow 

app: app.cc
	$(CXX) -o $@ $(LDFLAGS) $^ -m64 -fPIC -Wl,-zdefs -Wl,-znow -ldl

-include $(patsubst %.cc,%.d,$(wildcard *.cc))

AttachmentSize
plugin.tgz1.94 KB

Java AVL Tree

A non-recursive Java AVL Tree implementation. Performance is comparable (insertion slightly worse, search slightly better) with TreeSet of SUN JDK. By specializing the generic version for primitive types the ~2.5 performance boost can be achieved. The tree can be further optimized via height-independent balances update. This is not done for the sake of the code simplicity.

package net.bobah.containers.trees.avl;

import java.util.Iterator;

public class AvlTreeSet<T extends Comparable<T>> {

  public int height() { ... }

  public int size() { ... }

  public Iterator<T> iterator() { ... }

  public boolean insert(T value) { ... }

  public boolean remove(T value) { ... }

  public boolean contains(T value) { ... }

}

AttachmentSize
avltree.zip3.87 KB

Message Queue Event Loop Example (J2SE)

The code demonstrates the convenient and minimalistic implementation of the message queue event loop with restart capability.

/**
 * @author Vladimir Lysyy
 */

package net.bobah.concurrent.demo;

import java.util.concurrent.SynchronousQueue;

public class Runner implements Runnable {
  Thread thread = null;
  SynchronousQueue<Object> queue = new SynchronousQueue<Object>(true);

  private static void log(final String msg) {
    System.out.println(Thread.currentThread().getName() + " - " + msg);
  }

  @Override
  public void run() {
    log("thread function enter");
    try {
      for (; !Thread.interrupted();) {
        // Do something useful
        Object o = queue.take();
        log("processing [" + o + "]");
        // Let other threads do something useful
        Thread.yield();
      }
    }
    catch (InterruptedException e) {
      log("thread function interrupted");
    }
    log("thread function exit");
  }

  public synchronized void start() {
    if (thread == null) {
      log("starting thread");
      thread = new Thread(this);
      thread.start();
      log("thread started");
    }
  }

  public synchronized void stop() throws InterruptedException {
    if (thread != null) {
      if (thread.isAlive()) {
        log("stopping thread");
        thread.interrupt();
        thread.join();
        log("thread stopped");
      }
      thread = null;
    }
  }
 
  public void submit(Object o) throws InterruptedException {
    queue.put(o);
  }

  public static void main(String[] args) {
    try {
    Runner inst = new Runner();

    // starting a thread
    inst.start();
    // sending couple of messages
    inst.submit("message 1".intern());
    inst.submit("message 2".intern());
    Thread.sleep(500);
    // asynchronously stopping it while it's blocking on the queue
    inst.stop();

    // restarting a thread and
    inst.start();
    // sending another couple of messages
    inst.submit("message 3".intern());
    inst.submit("message 4".intern());
    // asynchronously stopping it while it's blocking on the queue
    Thread.sleep(500);
    inst.stop();
    } catch (Exception e) {
      e.printStackTrace();
    }
  }
}

Object factory in C++11

A constructor arguments caching object factory. Implementing it from scratch may be a good interview question for RVO, rvalue references, move semantics and smart pointers in C++11.
#include <memory>
#include <functional>
#include <utility>

#include <boost/functional/factory.hpp>

struct manufacturable {
    virtual ~manufacturable() {};
    virtual void api_method() const = 0;
};
using manufacturable_ptr = std::shared_ptr<struct manufacturable>;

struct factory {
    virtual ~factory() {}
    virtual manufacturable_ptr create() const = 0;
};
using factory_ptr = std::shared_ptr<struct factory>;

template <class T, class... Args>
struct factory_tpl final: factory
{
    std::function<std::shared_ptr<T>()> impl_;
    factory_tpl(Args&&... args):
        impl_(std::bind(boost::factory<std::shared_ptr<T>>(), std::forward<Args>(args)...))
    {}

    virtual manufacturable_ptr make() const override { return std::dynamic_pointer_cast<struct manufacturable>(impl_()); }
};

template <class T, class... Args>
factory_ptr make_factory(Args&&... args)
{
    return std::dynamic_pointer_cast<struct factory>(std::make_shared< factory_tpl<T, Args...> >(std::forward<Args>(args)...));
}

struct concrete_manufacturable final: manufacturable
{
    std::string const domain_;
    concrete_manufacturable(std::string const& id): domain_(domain) {}
    virtual void api_method() const override {}
}

int main()
{
    factory_ptr factory = make_factory<concrete_manufacturable>("demo");
    manufacturable_ptr object = factory->make();
    object->api_method();
    return 0;
}

Thread Pool Executor Example (J2SE)

The code demonstrates using the thread pool for tasks execution in J2SE, Sun documentation is here.
/**
 * @author Vladimir Lysyy
 */

package net.bobah.concurrent.demo;

import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
//import java.util.concurrent.ExecutorService;
//import java.util.concurrent.Executors;
import java.util.concurrent.RejectedExecutionException;
import java.util.concurrent.RejectedExecutionHandler;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

public class PoolDemo {
  private static class RejectedHandler implements RejectedExecutionHandler {
    @Override
    public void rejectedExecution(Runnable arg0, ThreadPoolExecutor arg1) {
      // TODO Auto-generated method stub
      System.err.println(Thread.currentThread().getName() + " execution rejected: " + arg0);     
    }
  }

  private static class Task implements Runnable {
    private static SimpleDateFormat fmt = new SimpleDateFormat("yyyy-MM-dd hh:mm:ss.SSS");
    private String name;
    private Date created;

    public Task(String name) {
      this.name = name;
      this.created = new Date();
    }
   
    @Override
    public void run() {
      final boolean wantOverflow = true;
      System.out.println(Thread.currentThread().getName() + " executing " + this);
      try {
        Thread.sleep(wantOverflow ? 50 : 10);
      } catch (InterruptedException e) {
        e.printStackTrace();
      }
      System.out.println(Thread.currentThread().getName() + " executed " + this);
    }

    @Override
    public String toString() {
      return name + ", created " + fmt.format(created);
    }

  }

  public static void main(String[] args) throws InterruptedException {
    final boolean wantExceptionOnReject = false;

//    // fixed pool, unlimited queue
//    ExecutorService service = Executors.newFixedThreadPool(10 /* size */);
//    ThreadPoolExecutor executor = (ThreadPoolExecutor) service;

    // fixed pool fixed queue
    BlockingQueue<Runnable> queue = new ArrayBlockingQueue<Runnable>(100, true);
    ThreadPoolExecutor executor = new ThreadPoolExecutor(
      10, // core size
      20, // max size
      1, // keep alive time
      TimeUnit.MINUTES, // keep alive time units
      queue // the queue to use
    );

    // set rejected execution handler
    // or catch exception from executor.execute (see below)
    if (!wantExceptionOnReject) executor.setRejectedExecutionHandler(new RejectedHandler());

    for(long i = 0; ; ++i) {
      Task t = new Task(String.valueOf(i));
      System.out.println(Thread.currentThread().getName() + " submitted " + t + ", queue size = " + executor.getQueue().size());
      try {
        executor.execute(t);
      } catch (RejectedExecutionException e) {
        // will be thrown if rejected execution handler
        // is not set with executor.setRejectedExecutionHandler
        e.printStackTrace();
      }
      Thread.sleep(1);
    }
  }
}

invokevirtual vs invokeinterface performance benchmark

A benchmark code to compare invokevirtual with invokeinterface performance. invokeinterface is 38% slower. If volatile modifier is removed from aand i (and it becomes possible for JVM to cache methods resolution results), the performance becomes comparable for both cases and the code runs ~10x faster than with volatile variables. A background in essence is "interfaces do not exist in runtime" so virtual dispatching can't be applied to interface methods. The idea is very well explained here. Below is a benchmark to compare invokevirtual with invokeinterface performance. Prevents JVM from optimizing out invokeinterface by declaring target object volatile.
public class InvokevirtualVsInvokeinterface {
  private static interface I {
    public int getInteger ();
  }

  private static class A implements I {
    public int getInteger () { return 0; }
  }

  private static class B extends A { }

  static volatile I i = new B();
  static volatile A a = new B();

  public static void main(String[] args) {
    {
      long tm1 = System.nanoTime();
      for (int k = 0; k < 100000000; ++k) {
        a.getInteger();
      }
      long tm2 = System.nanoTime();
      System.out.println("invokevirtual took " + (Math.abs(tm2 - tm1) / 1000) + " us");
    }

    {
      long tm1 = System.nanoTime();
      for (int k = 0; k < 100000000; ++k) {
        i.getInteger();
      }
      long tm2 = System.nanoTime();
      System.out.println("invokeinterface took " + (Math.abs(tm2 - tm1) / 1000) + " us");
    }

    // Output on Intel Xeon X5570 @ 2.93GHz:
    // invokevirtual took 41170 us
    // invokeinterface took 66305 us
  }
}
and the corresponding bytecode (the output of "javap -c")
Compiled from "InvokevirtualVsInvokeinterface.java"
public class InvokevirtualVsInvokeinterface extends java.lang.Object{
static volatile InvokevirtualVsInvokeinterface$I i;

static volatile InvokevirtualVsInvokeinterface$A a;

public InvokevirtualVsInvokeinterface();
  Code:
   0:	aload_0
   1:	invokespecial	#1; //Method java/lang/Object."<init>":()V
   4:	return

public static void main(java.lang.String[]);
  Code:
   0:	invokestatic	#2; //Method java/lang/System.nanoTime:()J
   3:	lstore_1
   4:	iconst_0
   5:	istore_3
   6:	iload_3
   7:	ldc	#3; //int 100000000
   9:	if_icmpge	25
   12:	getstatic	#4; //Field a:LInvokevirtualVsInvokeinterface$A;
   15:	invokevirtual	#5; //Method InvokevirtualVsInvokeinterface$A.getInteger:()I
   18:	pop
   19:	iinc	3, 1
   22:	goto	6
   25:	invokestatic	#2; //Method java/lang/System.nanoTime:()J
   28:	lstore_3
   29:	getstatic	#6; //Field java/lang/System.out:Ljava/io/PrintStream;
   32:	new	#7; //class java/lang/StringBuilder
   35:	dup
   36:	invokespecial	#8; //Method java/lang/StringBuilder."<init>":()V
   39:	ldc	#9; //String invokevirtual took 
   41:	invokevirtual	#10; //Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
   44:	lload_3
   45:	lload_1
   46:	lsub
   47:	invokestatic	#11; //Method java/lang/Math.abs:(J)J
   50:	ldc2_w	#12; //long 1000l
   53:	ldiv
   54:	invokevirtual	#14; //Method java/lang/StringBuilder.append:(J)Ljava/lang/StringBuilder;
   57:	ldc	#15; //String  us
   59:	invokevirtual	#10; //Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
   62:	invokevirtual	#16; //Method java/lang/StringBuilder.toString:()Ljava/lang/String;
   65:	invokevirtual	#17; //Method java/io/PrintStream.println:(Ljava/lang/String;)V
   68:	invokestatic	#2; //Method java/lang/System.nanoTime:()J
   71:	lstore_1
   72:	iconst_0
   73:	istore_3
   74:	iload_3
   75:	ldc	#3; //int 100000000
   77:	if_icmpge	95
   80:	getstatic	#18; //Field i:LInvokevirtualVsInvokeinterface$I;
   83:	invokeinterface	#19,  1; //InterfaceMethod InvokevirtualVsInvokeinterface$I.getInteger:()I
   88:	pop
   89:	iinc	3, 1
   92:	goto	74
   95:	invokestatic	#2; //Method java/lang/System.nanoTime:()J
   98:	lstore_3
   99:	getstatic	#6; //Field java/lang/System.out:Ljava/io/PrintStream;
   102:	new	#7; //class java/lang/StringBuilder
   105:	dup
   106:	invokespecial	#8; //Method java/lang/StringBuilder."<init>":()V
   109:	ldc	#20; //String invokeinterface took 
   111:	invokevirtual	#10; //Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
   114:	lload_3
   115:	lload_1
   116:	lsub
   117:	invokestatic	#11; //Method java/lang/Math.abs:(J)J
   120:	ldc2_w	#12; //long 1000l
   123:	ldiv
   124:	invokevirtual	#14; //Method java/lang/StringBuilder.append:(J)Ljava/lang/StringBuilder;
   127:	ldc	#15; //String  us
   129:	invokevirtual	#10; //Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
   132:	invokevirtual	#16; //Method java/lang/StringBuilder.toString:()Ljava/lang/String;
   135:	invokevirtual	#17; //Method java/io/PrintStream.println:(Ljava/lang/String;)V
   138:	return

static {};
  Code:
   0:	new	#21; //class InvokevirtualVsInvokeinterface$B
   3:	dup
   4:	aconst_null
   5:	invokespecial	#22; //Method InvokevirtualVsInvokeinterface$B."<init>":(LInvokevirtualVsInvokeinterface$1;)V
   8:	putstatic	#18; //Field i:LInvokevirtualVsInvokeinterface$I;
   11:	new	#21; //class InvokevirtualVsInvokeinterface$B
   14:	dup
   15:	aconst_null
   16:	invokespecial	#22; //Method InvokevirtualVsInvokeinterface$B."<init>":(LInvokevirtualVsInvokeinterface$1;)V
   19:	putstatic	#4; //Field a:LInvokevirtualVsInvokeinterface$A;
   22:	return

}

Process

Everyone has his know-how's for organizing a workspace and a working process for the best performance and convenience. I'm sharing my ideas and would be happy to hear your critics and improvement suggestions.

Raspberry Pi

Send an E-mail when LAN or WAN IP changes

My Raspberry Pi is WI-FI DHCP behind the WAN facing firewall. I wanted it to send me an e-mail every time its IP or WAN IP of the firewall change and below is what I came up with. I have used the automation API of whatismyip.com to determine the WAN IP of the firewall and the output of /bin/hostname -I
#!/bin/bash

set -e
set -u

read newlanip < <(/bin/hostname -I)
newwanip=$(/usr/bin/curl -s -m20 -A 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20100101 Firefox/12.0' http://automation.whatismyip.com/n09230945.asp)

echo "new IPs: [$newlanip] [$newwanip]"

if [[ ! -s ${0%.sh}.myip ]]; then
  echo "0.0.0.0 0.0.0.0" > ${0%.sh}.myip
  echo "no saved IP, generated ${0%.sh}.myip"
fi

read oldlanip oldwanip < ${0%.sh}.myip
echo "old IPs: [$oldlanip] [$oldwanip]"

echo "$newlanip $newwanip" > ${0%.sh}.myip
if [[ $newlanip != $oldlanip || $newwanip != $oldwanip ]]; then
  echo "IP changed: [$newlanip] [$newwanip]"
  mail -s "My IPs: LAN:[$newlanip] WAN:[$newwanip]" mail'@'bobah.net < /dev/null
fi

Unix

Random notes about Unix environment configuraion

Keyboard Layout Shortcut in Fluxbox

After having tired of Ubuntu overweighted window manager on my EeePC 1005HA I have decided to return to the old good Fluxbox. As a bi-lingual user I absolutely needed a handy way of switching keyboard layout, ideally similar to that of M$. Unfortunately (or fortunately) minimalistic defaults of the Fluxbox did not have anything to be used out of the box, so I had to find the solution myself. Here is what I came up wtih after having read this and this:
# ~/.fluxbox/keys
# Switch between us and ru keyboards on alt + left shift
# Mod1 is ALT, Shift_L is what printed by xev upon left shift key press
Mod1 Shift_L :Exec if [[ $(setxkbmap -query|awk '/layout:/ {print $2}') == 'us' ]]; then setxkbmap ru; else setxkbmap us; fi