_.at

ElasticSearch Curator Dependencies

last change: 2019-04-11

After quite some time figuring out the dependencies for ElasticSearch Curator 5.6 here is the list for offline installation:

boto3-1.9.130.tar.gz
botocore-1.12.130.tar.gz
python-certifi-2018.10.15.tar.gz
chardet-3.0.4.tar.gz
python-dateutil-2.8.0.tar.gz
click-6.7.tar.gz
requests-2.21.0.tar.gz
curator-5.6.0.tar.gz
requests-aws4auth-0.9.tar.gz
docutils-0.14.tar.gz
s3transfer-0.2.0.tar.gz
elasticsearch-py-6.3.1.tar.gz
setuptools_scm-3.2.0.tar.gz
ez_setup.py
six-1.12.0.tar.gz
idna-2.8.tar.gz
urllib3-1.24.1.tar.gz
jmespath-0.9.4.tar.gz
voluptuous-0.11.5.tar.gz
PyYAML

permlink: _.at: ElasticSearch Curator Dependencies

Delete all data in a MongoDB collection

last change: 2019-03-01

# mongo <dbname>
> use <dbname>
> db.<collection name>.remove({})
> db.repairDatabase()
> db.compact()

permlink: _.at: Delete all data in a MongoDB collection

Sublime Text commandline utility on macOS

last change: 2019-01-17

sudo ln -s "/Applications/Sublime Text.app/Contents/SharedSupport/bin/subl" /usr/local/bin/subl

permlink: _.at: Sublime Text commandline utility on macOS

Check SSL server certificate contents

last change: 2018-06-15

echo | openssl s_client -showcerts -servername <server name> -connect <host name>:443 2>/dev/null | openssl x509 -inform pem -noout -text

server name is used to add SNI compatible headers to the request. host name is the name of the machine and can be the same as server name.

Links:
ServerFault: Displaying a remote SSL certificate details using CLI tools

permlink: _.at: Check SSL server certificate contents

Git changes in branch

last change: 2018-06-14

See the changes in a branch since branching:
git checkout <branch name>
base=`git merge-base master HEAD`
git diff --name-only $base HEAD

Links:
Stackoverflow: git - changes to branch since created

permlink: _.at: Git changes in branch

Git branch cleanup

last change: 2018-07-04

List branches:
git branches

Delete branch from remote 'origin':
git push -d origin <branch name>

Delete local branch:
git branch -d <branch name>

Rename current local branch:
git branch -n <new name>

Rename some local branch:
git branch -m <old name> <new name>

Rename remote branch after renaming local branch:
git push origin :<old name>
git push --set-upstream origin <new name>

Links:
Stackoverflow: How do I delete a Git branch both locally and remotely

permlink: _.at: Git branch cleanup

Postfix TLS

last change: 2016-08-09

/etc/postfix/main.cf:
# TLS parameters
smtp_tls_security_level = encrypt
smtpd_tls_security_level = encrypt
smtpd_tls_auth_only = no
smtp_tls_note_starttls_offer = yes
smtpd_tls_key_file=/etc/ssl/private/smtpd.key
smtpd_tls_cert_file=/etc/ssl/certs/smtpd.crt
smtpd_tls_CAfile = /etc/ssl/certs/cacert.pem
smtpd_tls_loglevel = 1
smtpd_tls_received_header = yes
smtpd_tls_session_cache_timeout = 3600s
tls_random_source = dev:/dev/urandom
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtpd_tls_mandatory_ciphers = high
smtpd_tls_mandatory_exclude_ciphers = aNULL, MD5
smtpd_tls_mandatory_protocols = TLSv1
# Also available with Postfix >= 2.5:
#smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3

Test:
openssl s_client -starttls smtp -CApath /etc/ssl/certs/cacert.pem -connect localhost:25 -ssl3
openssl s_client -starttls smtp -CApath /etc/ssl/certs/cacert.pem -connect localhost:25 -ssl2

Links:
How to check TLS connections with openssl s_client

permlink: _.at: Postfix TLS

smbfs mount ignoring file_mode / dir_mode

last change: 2014-05-09

In case a mount using smbfs to mount a SAMBA share ignores the file_mode and dir_mode settings try adding the option 'nounix':

//samba-server.local.domain/home /remote-home smbfs username=sambauser,password=sambapassword,uid=localusermapping,gid=localgroupmapping,file_mode=0775,dir_mode=0775,rw,nounix 1 2

of course substituting the correct values for your configuration:
username: the remote SAMBA user account
password: the remote SAMBA password
uid: the local user to map the files to (i.e. local owner)
gid: local group to map to
file_mode, dir_mode: the modes for files resp. directories to use locally

permlink: _.at: smbfs mount ignoring file_mode / dir_mode

Creating SSL Keys and CSRs

last change: 2014-04-15

Create new private key and CSR:
openssl req -new -nodes -keyout server.key -out server.csr -newkey rsa:4096

Show contents of CSR:
openssl req -in server.csr -text

permlink: _.at: Creating SSL Keys and CSRs

Bash: argument list too long

last change: 2014-04-07

That moment of truth when you want to clean out a filled up directory and bash reassuringly tells you:

/bin/rm: cannot execute [Argument list too long]

Use find and xargs:
find . -name "*.log" -print0 | xargs -0 rm

but beware that this works recursively. Try "-maxdepth 1" to limit to the current directory.

permlink: _.at: Bash: argument list too long

Manual SMTP Relay Check

last change: 2014-03-19

Have a simple conversation with your SMTP server using telnet to port 25:

--> HELO youthostname
<-- 250 smtpservername
--> MAIL FROM: xx@xyz.xx
<-- 250 Ok
--> RCPT TO: <xx@xyz.xx> (place recipient address here)
<-- 554 <xx@xyz.xx>: Relay access denied

Links:
Hetzner Wiki: Manual Relay Check

permlink: _.at: Manual SMTP Relay Check

HTML5 <video> tag in IE9

last change: 2013-11-27

If IE9 shows a warning that it does not support the <video> tag within an HTML5 document (with the correct DOCTYPE add this to the <head>:

permlink: _.at: HTML5 <video> tag in IE9

OpenSSH / Ubuntu PAM Headers missing

last change: 2013-11-22

When compiling OpenSSH on Ubuntu you might get the error message "PAM headers not found". Try installing the missing header files:

sudo apt-get install libpam0g-dev

and retry
./configure --prefix=/usr --sysconfdir=/etc/ssh --with-pam

permlink: _.at: OpenSSH / Ubuntu PAM Headers missing

Expiration Date of a PKCS#12 certificate

last change: 2013-11-19

Via the source linked to below...

openssl pkcs12 -in certificate.p12 -out tempcrt.pem
openssl x509 -in tempcrt.pem -noout -enddate

Links:
BSDSupport.org: How do I determine the expiration date of a p12 certificate?

permlink: _.at: Expiration Date of a PKCS#12 certificate

Commons Logging

last change: 2013-06-18

Configure Commons Logging to use Log4J: create the file commons-logging.properties with the following content:
org.apache.commons.logging.Log=org.apache.commons.logging.impl.Log4JLogger

and put it in the CLASSPATH

permlink: _.at: Commons Logging

Compile Apache 2.4

last change: 2013-05-17

Install packages libpcre3 and libpcre3-dev

mkdir download compile
cd compile
tar xzvf ../download/httpd-2.4.4.tar.gz
cd httpd-2.4.4/srclib
tar xzvf ../../../download/apr-util-1.5.2.tar.gz
mv apr-util-1.5.2/ apr-util
tar xzvf ../../../download/apr-1.4.6.tar.gz
mv apr-1.4.6/ apr
cd ..
./configure '--enable-so' \
'--enable-ssl' \
'--enable-rewrite=shared' \
'--enable-headers=shared' \
'--enable-proxy=shared' \
'--enable-proxy-balancer=shared' \
--with-included-apr

Links:
Apache HTTPd 2.4 Download
Apache APR Download

permlink: _.at: Compile Apache 2.4

Problems with configure

last change: 2013-05-17

When configure exits with the message "configure: error: cannot run C compiled programs." under Ubuntu reinstallation of libc-dev might help:

aptitude update
apt-get install --reinstall libc6-dev
apt-get install libc6-dev-i386

permlink: _.at: Problems with configure

Installing pdsh on MacOS X

last change: 2013-05-14

How to compile pdsh, the distributed parallel shell, on MacOS X

Download and compile readline:
cd readline-6.2
MACOSX_DEPLOYMENT_TARGET=10.8 ARCHFLAGS="-arch x86_64" \
./configure --prefix=/usr/local --enable-shared
# regardign error about -dynamiclib: seehttp://www.iamseree.com/application-development/readline-6-2-make-error-in-mac-os-x-lion
cd shlib
sed -e 's/-dynamic/-dynamiclib/' Makefile > Makefile.osx
mv Makefile.osx Makefile
cd ..
#
make
make && sudo make install

Download and compile pdsh:
cd pdsh-2.29
./configure --with-ssh --with-readline --without-rsh
make && sudo make install

Links:
GNU Readline library
PDSH

permlink: _.at: Installing pdsh on MacOS X

Changing the senders hostname for Java Mail

last change: 2013-03-27

Add the following property to your mail configuration, substituting the correct name for my.mail.hostname.com:

properties.put("mail.smtp.localhost", "my.mail.hostname.com");

permlink: _.at: Changing the senders hostname for Java Mail

Rails with JRuby on MacOS X

last change: 2012-07-25

Install Rails Gem:
$ sudo jruby -S gem install rails

If using a proxy either add the corresponding command line flag or edit ~/.gemrc (see below):
$ sudo jruby -S gem install rails --http-proxyhttp://proxy.company.com:8080

Install JRuby SSL support:
$ sudo jruby -S gem install jruby-openssl

Create a dummy project to install the missing gems:
$ sudo jruby -S rails new RailsTest

When using a proxy you can add the hostname to ~/.gemrc like this:
gem: --http-proxyhttp://proxy.company.com:8080
bundle: --http-proxyhttp://proxy.company.com:8080
bundler: --http-proxy http://proxy.company.com:8080

permlink: _.at: Rails with JRuby on MacOS X

Import PKCS#12 certificates into a Java Keystore

last change: 2012-06-04

keytool -importkeystore \
-deststorepass KEYSTORE_PASSWORD \
-destkeypass KEYSTORE_PASSWORD \
-destkeystore KEYSTORE_FILENAME \
-srckeystore INPUTFILE \
-srcstoretype PKCS12 \
-srcstorepass INPUTFILE_PASSWORD \
-alias 1

permlink: _.at: Import PKCS#12 certificates into a Java Keystore

Convert secsh Public Key format

last change: 2011-11-14

Convert a file in secsh-format (e.g. coming out of Putty) to a format usable with OpenSSH:

ssh-keygen -i -f putty-key.pub >openssh-key.pub

Links:
SSH Implementation Interoperability

permlink: _.at: Convert secsh Public Key format

Preprocess Apache Logs for Evaluation

last change: 2011-11-09

This simple shellscript converts standard Apache common logfiles to a CSV format (note the tabs in the replacement pattern between each column):

#!/bin/sh
echo "IP\tTag\tMonat\tJahr\tUhrzeit\tURL\tStatus\tTransfer"
# 1: IP
# 2: day
# 3: month
# 4: year
# 5: time
# 6: url
# 7: protocol
# 8: status code
# 9: transfer in bytes

cat $1 | \
sed -E 's/(.*) \- \- \[([0-9]*)\/([A-Z][a-z]*)\/([0-9]*):(.*) \+[0-9]*\] "GET (.*) HTTP\/(.*)" +([0-9]*) ([0-9]*) "(.*)" .*/\1 \2 \3 \4 \5 \6 \7 \9/g'

permlink: _.at: Preprocess Apache Logs for Evaluation

FOP Font installation

last change: 2011-06-30

Put Font files (TTF or OTF) in same directory as the XSL file, create a file called fop-config.xml with this content:

Alternatively put font files under <JAVA_HOME>/jre/lib/fonts/

permlink: _.at: FOP Font installation

XSL-FO hyphenation library

last change: 2011-06-30

To enable hyphenation when using Apache FOP 1.x use

<fo:block margin-left="0mm" font-size="10pt" text-align="justify" language="de" hyphenate="true">
<xsl:value-of select="."/>
</fo:block>

after installing the OFFO hyphenation library.

Links:
FOP XML Hyphenation Patterns
Objects for Formatting Objects Binary download

permlink: _.at: XSL-FO hyphenation library

Image format conversion

last change: 2011-06-30

From (CMYC) EPS or Adobe Illustrator etc. to JPG using ImageMagick:
convert -density 300 -colorspace RGB inputfile.ai -trim outputfile.jpg

Or using transparency:
convert -density 300 -colorspace RGB inputfile.ai -trim +repage -background transparent -alpha background PNG32:outputfile.png

From EPS to SVG using pstoedit:
pstoedit -f plot-svg inputfile.eps outputfile.svg
This uses a shareware plugin for pstoedit to enable SVG output.

Links:
ImageMagick
pstoedit
pstoedit Plugin license

permlink: _.at: Image format conversion

Installing ImageMagick PHP extension on Ubuntu

last change: 2011-03-18

# apt-get install libmagickwand-dev libmagickcore-dev

# pecl install imagick

permlink: _.at: Installing ImageMagick PHP extension on Ubuntu

Installing mcrypt PHP extension on Ubuntu

last change: 2011-03-18

# apt-get install libmcrypt-dev

# cd php-5.x.x/ext/mcrypt
# phpize
# aclocal
# ./configure
# make && make install

Add 'extension=mcrypt.so' to php.ini.

permlink: _.at: Installing mcrypt PHP extension on Ubuntu

HowTo: Resize an EXT4-formatted Linux LVM Volume in a virtual machine

last change: 2011-02-17

1. Resize the volume using VMWare VM-Manager. The size field is greyed out if there are any snapshots defined... you'll have to delete them.

2. Start the virtual machine.

3. Use cfdisk to create a new logical partition in the free space of the virtual disk. Use type 8e - Linux LVM. Note: fdisk did not see the free space on the device. The new partition is /deb/sda6 in this example.

4. REBOOT the virtual machine. In our tests the change to the partition table was not visible after exiting cfdisk.
# reboot

5. Create a physical volume:
# pvcreate /dev/sda6
Physical volume "/dev/sda6" successfully created

6. Use lvdisplay to look up the name of the volume group currently in use:
# lvdisplay
--- Logical volume ---
LV Name                /dev/mongo1/root
VG Name                mongo1
LV UUID                wi9dRo-fS0C-xSCd-fbei-el9R-vpVi-0QEiIR
LV Write Access        read/write
LV Status              available
# open                 1
LV Size                9,29 GiB
Current LE             2379
Segments               1
Allocation             inherit
Read ahead sectors     auto
- currently set to     256
Block device           251:0

7. Add the physical volume to the volume group found in the field 'VG Name':
# vgextend mongo1 /dev/sda6
Volume group "mongo1" successfully extended

8. Now you can try to extend the existing logical volume (as named in field 'LV Name' above). If the given size increase (20GB in this example) is more than is available in the disk group, the error message will indicate the number of blocks available:
# lvextend -L +20G /dev/mapper/mongo1-root
Extending logical volume root to 29,29 GiB
Insufficient free space: 5120 extents needed, but only 5119 available

Retry with the number of blocks taken from this error message:
# lvextend -l +5119 /dev/mapper/mongo1-root
Extending logical volume root to 29,00 GiB
Logical volume root successfully resized

9. Resize the filesystem (sorry for the german output -- this is the only command that used the locale setting...):
# resize2fs /dev/mapper/mongo1-root
resize2fs 1.41.11 (14-Mar-2010)
Das Dateisystem auf /dev/mapper/mongo1-root ist auf / eingehängt; Online-Grössenveränderung nötig
old desc_blocks = 1, new_desc_blocks = 2
Führe eine Online-Grössenänderung von /dev/mapper/mongo1-root auf 5241856 (4k) Blöcke durch.
Das Dateisystem auf /dev/mapper/mongo1-root ist nun 5241856 Blöcke gross.

Links:
LVM Logical Volume Manager

permlink: _.at: HowTo: Resize an EXT4-formatted Linux LVM Volume in a virtual machine

Amazon

last change: 2010-07-09

Amazon seems to try to act as a Copyright Cop recently: the large view of book covers is protected by futile measures to prevent saving these images. A simple look into the local browser cache (assisted by Firefox through means Page Info -> Media) reveals everything needed...

permlink: _.at: Amazon

Helpdesk

last change: 2009-11-16

How to complain about a malfunctioning vacuum cleaner:
"This thing sucks. That is... actually it doesn't."

permlink: _.at: Helpdesk

Really bad...

last change: 2009-10-23

Why do Ubuntu releases have to have these really idiotic names? Like 'Karmic Koala'... I'm just waiting for a version going by 'Sterioidal Snail' or something like that...

permlink: _.at: Really bad...

SCO leaving the Past behind

last change: 2009-10-20

Seems as his last crusade is over: Darl McBride is history. At least as SCOs CEO, they kicked him in an effort to recover from bankruptcy. Good move...

Links:
The Register: SCO boots boss McBride

permlink: _.at: SCO leaving the Past behind

Blackberry Desktop for Mac

last change: 2009-10-02

Seems already to be available for Download...

Links:
Blackberry Desktop for Mac

permlink: _.at: Blackberry Desktop for Mac

Mercurial on MacOS X

last change: 2009-07-22

When installing Mercurial from source using 'make install' it copies its support files to the directory /usr/local/lib/python2.5/site-packages/
Running 'hg debuginstall' gives an error like this:
abort: couldn't find mercurial libraries in [...]

To correct this do the following:

sudo mv /usr/local/lib/python2.5/site-packages/hgext/ /Library/Python/2.5/site-packages/
sudo mv /usr/local/lib/python2.5/site-packages/mercurial-1.3-py2.5.egg-info /Library/Python/2.5/site-packages/
sudo mv /usr/local/lib/python2.5/site-packages/mercurial/ /Library/Python/2.5/site-packages/

'hg debuginstall' should now run fine.

2009/07/22:
Added link to Mercurial binaries for Mac. Version 1.3 is already present there.

Links:
Mercurial Binaries for Windows and Mac

permlink: _.at: Mercurial on MacOS X

Mercurial default push repository

last change: 2009-07-22

To set the default repository to push changes to open .hg/hgrc within the project and add the following section (replacing the mercurial-server and the project-name parts of the URL appropriatly):

[paths]
default = ssh://hg@<mercurial-server>/<project-name>

Note that this default entry is created for you each time you clone a remote repository.

You can add more short names to this section to use them in push and pull commands, just replace 'default' with the name like this:

ted = ssh://hg@teds-machine/teds-project

and use them:

hg push ted
hg pull ted

permlink: _.at: Mercurial default push repository

Gitosis Problem

last change: 2009-07-20

As gitosis suddenly told me the following when doing a push:

$ git push
Traceback (most recent call last):
File "/usr/bin/gitosis-serve", line 8, in <module>
load_entry_point('gitosis==0.2', 'console_scripts', 'gitosis-serve')()
File "/usr/lib/python2.5/site-packages/gitosis-0.2-py2.5.egg/gitosis/app.py", line 24, in run
return app.main()
File "/usr/lib/python2.5/site-packages/gitosis-0.2-py2.5.egg/gitosis/app.py", line 38, in main
self.handle_args(parser, cfg, options, args)
File "/usr/lib/python2.5/site-packages/gitosis-0.2-py2.5.egg/gitosis/serve.py", line 204, in handle_args
os.execvp('git', ['git', 'shell', '-c', newcmd])
File "/usr/lib/python2.5/os.py", line 354, in execvp
_execvpe(file, args)
File "/usr/lib/python2.5/os.py", line 392, in _execvpe
func(fullname, *argrest)
OSError: [Errno 2] No such file or directory
fatal: The remote end hung up unexpectedly

The reason was taht unexplicably it somehow forgot a path (git is installed unter /usr/local on that server). Adding a symlink on the server

ln -s /usr/local/bin/git /usr/bin

solved the problem. Adding the PATH variable to the users homedirectories .bashrc (and/or .profile) did not help.

permlink: _.at: Gitosis Problem

PostgreSQL 8.4

last change: 2009-07-07

Interesting additions to the administration tool suite (from the press release):

* Parallel Database Restore, speeding up recovery from backup up to 8 times
* Per-Column Permissions, allowing more granular control of sensitive data
* Per-database Collation Support, making PostgreSQL more useful in multi-lingual environments
* In-place Upgrades through pg_migrator (beta), enabling upgrades from 8.3 to 8.4 without extensive downtime
* New Query Monitoring Tools, giving administrators more insight into query activity
* Greatly Reduced VACUUM Overhead through the Visibility Map
* New Monitoring Tools for current queries, query load and deadlocks

Links:
PostgreSQL 8.4 released

permlink: _.at: PostgreSQL 8.4

Database maintainance

last change: 2009-07-07

Automate Database maintainance: this tool keeps track which update scripts have already been applied to a database to help keep development and production database updates in sync.

Links:
DBMaintain.org

permlink: _.at: Database maintainance

Lift Webapplication Framework

last change: 2009-06-08

As each language needs a webapplication framework these days, here is Scalas take: Lift. Combining some ideas from Rails (such as sensible configuration defaults) with the basic abilities of the functional-objectorientied hybrid language Scala (utilizing for example the Actor library) this framework sounds quite compelling.

Links:
Lift Webframework

permlink: _.at: Lift Webapplication Framework

Unix, Windows and programming language timelines

last change: 2009-05-27

As the title says: a graphical timeline of the evolution of Unix and related systems, of Windows and of sme programming languages. Interesting cubicle wallpaper...

Links:
Éric Lévénez's site

permlink: _.at: Unix, Windows and programming language timelines

Sphinx fulltext search engine

last change: 2009-05-11

Nice article over at The Register outlining the story of Sphinx, the open source fulltext search engine powering craigs lists as well as other high profile sites.
From the article:
'... the Apache Foundation, of course, widely known as a cruel experiment to see what happens when bureaucrats do open source.'
Full ACK.

Links:
The Register: Sphinx - text search The Pirate Bay way

permlink: _.at: Sphinx fulltext search engine

Standards...

last change: 2009-05-11

... help you differentiate your product from other vendors by eliminating the chance of unintended compatibility: you know what to avoid...

permlink: _.at: Standards...

Will Oracle kill MySQL?

last change: 2009-05-06

The corrent answer as of the articlet linked below: who cares? Could not say this any better...

From the article:
"Years ago, it [the open source community] taught Larry Ellison hate. Now, Ellison is teaching it fear."

Links:
The Register: Will Oracle kill MySQL? Who cares?

permlink: _.at: Will Oracle kill MySQL?

IP Subnet Calculator

last change: 2009-05-04

OK, the 10.000. instance of a solution to this minor problem. But nevertheless usable.

Links:
IP Subnet Calculator

permlink: _.at: IP Subnet Calculator

Gitosis

last change: 2009-04-28

Gitosis Error when trying to add another repository: Repository read access denied

more...

all articles represent the sole opinion of their respective author. all content comes without any warranty for correctnes, despite due diligence.