阿福的空虛世界

使用govendor管理golang packages

因為工作使用Go當backend， 想當然一開始要把環境建構起來，
當時的專案裡，有一份環境建構的文件，
照著文件做，結果環境卻build failed !?

因為都是用 go get github.com/xxxx/packages 下載，
因為某些套件的更新，導致一些原本可以用的method變得不相容，
所以需要一個package管理的套件。

從Go的github wiki有列出幾個管理套件，
有試過govender以及glide，感覺govender比較適合我。

從Go 1.6以後就有支援vendor的功能，1.5可以藉由GO15VENDOREXPERIMENT=1的環境變數來實現vendor功能。

廢話有點多，來點實際的範例吧！

1. 從 https://github.com/kilfu0701/go-vendor-example 下載

git clone https://github.com/kilfu0701/go-vendor-example.git
cd go-vendor-example
export GOPATH=`pwd`
export PATH=$GOPATH/bin:$PATH

2. 安裝govender

go get -u github.com/kardianos/govendor
rm -rf src/github.com src/golang.org                # after install govender, remove "src/github.com" & "src/golang.org"

3. 安裝example需要的packages

cd src/app
govendor sync

4. 回到example主目錄 執行測試

cd $GOPATH
go run src/main.go

打開瀏覽器 http://localhost:8081/

govender github
Goalng套件管理wiki

NodeJS - domain-parser

剛好想要這個功能，但是沒有現成的
有找到php版本，不過判斷的情況不是很完美
只好自己寫一個 XD

domain-parser
https://www.npmjs.org/package/domain-parser

Version: 0.0.5 (2014/10/29)

Source Code on Github
https://github.com/kilfu0701/node-domain-parser

npm install domain-parser

var dp = require('domain-parser');

var result = dp('www.google.com.tw').domainName;
console.log(result);
// google.com.tw

result = dp('direct.jp-bank.japanpost.jp').domainName;
console.log(result);
// japanpost.jp

非chrome web store的套件被強制停用

http://chrome.blogspot.tw/2014/05/protecting-chrome-users-from-malicious.html


今天打開chrome之後，發現自己寫的套件被停用了

後來查了資料才發現，google把 不是從web sotre上安裝的套件 都強制停用 QQ

看來要把自己寫的套件(外掛)，放到store拉

sqlalchemy 輸出json格式

剛好有人發問，寫了一個範例，

做個記錄一下，也許以後自己會用到 XD

http://pastie.org/8595769

# -*- coding: utf-8 -*-
import hashlib
import sqlalchemy
from sqlalchemy import Column, Integer, String, ForeignKey, DateTime, Table
from sqlalchemy.schema import MetaData
from sqlalchemy.orm import relationship
from sqlalchemy.orm import Bundle
from sqlalchemy.orm import lazyload
from sqlalchemy.orm.session import Session, sessionmaker
from sqlalchemy.ext.declarative import declarative_base

engine = sqlalchemy.create_engine('mysql+mysqldb://user:passwd@host/your_db?charset=utf8')
connection = engine.connect()
session = Session(bind=connection)

my_declarative_base = lambda cls: declarative_base(cls=cls)

@my_declarative_base
class Base(object):
    @property
    def columns(self):
        return [ c.name for c in self.__table__.columns ]

    @property
    def columnitems(self):
        return dict([ (c, getattr(self, c)) for c in self.columns ])

    def __repr__(self):
        return u'{}({})'.format(self.__class__.__name__, self.columnitems)

    def to_json(self):
        return self.columnitems

    def rec_json(self, key_str):
        result = self.columnitems
        result[key_str] = getattr(self, key_str).columnitems

        return result
        
class User(Base):
    __tablename__ = 'user'
    __table_args__ = {
        'mysql_engine': 'InnoDB',
        'mysql_charset': 'utf8'
    }

    id = Column(Integer, primary_key=True)
    name = Column(String(64))
    username = Column(String(64))
    password = Column(String(256))
    address_id = Column(Integer, ForeignKey('user_address.id'))
    address = relationship("Address", lazy="joined")

    def __init__(self, name, username, password):
        self.name = name
        self.username = username
        self.password = hashlib.sha1(password).hexdigest()

    def __repr__(self):
       return u"[User('%s','%s', '%s')]" % \
           (self.name, self.username, self.password)


class Address(Base):
    __tablename__ = 'user_address'
    __table_args__ = {
        'mysql_engine': 'InnoDB',
        'mysql_charset': 'utf8'
    }

    id = Column(Integer, primary_key=True)
    address = Column(String(256), nullable=False)

    def __init__(self, address):
        self.address = address

    def __repr__(self):
        return u"[Address('%s')]" % self.address
        
        

# insert test data...
"""
Base.metadata.create_all(engine)

user_1 = User("user1", "username1", "password_1")
user_2 = User("user2", "username2", "password_2")
address_1 = Address(u'臺灣凱達格蘭大道')
user_1.address = address_1
user_2.address = address_1
session.add(user_1)
session.add(user_2)
session.commit()
"""

for r in session.query(User).options(lazyload('address')):
    print r.rec_json('address')


"""
Output Result:

{'username': 'username1', 'address_id': 17L, 'name': 'user1', 'address': {'id': 17L, 'address': u'\u81fa\u7063\u51f1\u9054\u683c\u862d\u5927\u9053'},
 'password': '3833b3a1c69cf71a31d86cb5bb4d3866789b4d1e', 'id': 33L}
{'username': 'username2', 'address_id': 17L, 'name': 'user2', 'address': {'id': 17L, 'address': u'\u81fa\u7063\u51f1\u9054\u683c\u862d\u5927\u9053'},
 'password': '148dfdc3c539d35004cb808ca84e17ff962af744', 'id': 34L}

"""

1. http://chimerhapsody.blogspot.tw/2013/09/python-sqlalchemy-orm-part-3-end.html
2. http://docs.sqlalchemy.org/en/rel_0_9/orm/relationships.html
3. http://blogs.gnome.org/danni/2013/03/07/generating-json-from-sqlalchemy-objects/

安裝Solr 4.6

距離上次寫的安裝設定很遙遠，而且那時候用的還是比較舊的版本，

就趁這次重新安裝的時候，順便寫一篇給自己以後來用 XD


這邊我是都把所有東西放到 /opt/ 底下，

依個人需求自行調整。

1. 下載 solr 4.6

cd /opt/

wget http://ftp.tc.edu.tw/pub/Apache/lucene/solr/4.6.0/solr-4.6.0.tgz
tar zxvf solr-4.6.0.tgz

2. 下載 java sdk (http://www.oracle.com/technetwork/java/javase/downloads/index.html)

tar zxvf jdk-7u45-linux-i586.gz

3. 複製solr-4.6/example到/opt/solr/

cp -pr /opt/solr-4.6.0/example/ /opt/solr/

4. 啟動solr
方法1:

cd /opt/solr/
java -Dsolr.solr.home=multicore -jar start.jar

方法2:

wget https://gist.github.com/Polzme/7367523/raw/9ccf8b040b70eca214497dc6ee2b23e9abb0b8ef/solr -O /etc/init.d/solr
chmod +x /etc/init.d/solr

# 依個人需求，修改java的路徑
vim /etc/init.d/solr

#DAEMON=/usr/bin/java
DAEMON=/opt/jdk1.7.0_45/bin/java

service solr start

5. 檢查是否已經啟動?

http://localhost:8983/solr/

6. 設定開機的時候 啟動solr

update-rc.d solr defaults

7. 設定iptables

只允許localhost access 8983的port

iptables -A INPUT -p tcp -s localhost --dport 8983 -j ACCEPT
iptables -A INPUT -p tcp --dport 8983 -j DROP

1. http://apache.osuosl.org/lucene/solr/ref-guide/apache-solr-ref-guide-4.6.pdf
2. http://wiki.apache.org/solr/SolrTomcat
3. https://coderwall.com/p/0s3flw
4. https://cwiki.apache.org/confluence/display/solr/Running+Solr+on+Tomcat
5. https://gist.github.com/Polzme/7367523/

Export data to a CSV file with Django

單純因為會用到，所以就剛好寫到這個功能

想說，以後可能在別的公司or自己的server會用到，就順便整理一份範例。

https://gist.github.com/kilfu0701/7332112

# in some views.py, to serve a csv file in Django.
# -*- coding: utf-8 -*-
import csv

from django.http import HttpResponse

def donwload_csv(request):
    # change export csv formatting by request parameter. ( url?encode=utf-8 )
    encode = request.GET.get('encode', 'big5')
 
    response = HttpResponse(content_type='text/csv')
    response['Content-Disposition'] = 'attachment; filename="light_member.csv"'
 
    writer = csv.writer(response, delimiter=',', quotechar='"')
    header = [u'活動名稱', u'姓名', u'Email', u'會員身分', u'答案', u'活動登入日期', u'註冊日期', u'Device']
    writer.writerow([x.encode(encode) for x in header])
    # force export column by text format.
    # http://stackoverflow.com/questions/165042/stop-excel-from-automatically-converting-certain-text-values-to-dates
    deli = u'="{0}"'
 
    ## result['data'], some data from databases.
    #result = {}
    #result['data'] = list(aModel.objects.filter(**ftr).values(*fds))
    for i in result['data']:
        role = u'New' if i['new'] == 1 else u'Exist'
 
        raw_row = [i['plan'], i['name'], i['email'], role, i['option'], str(i['timestamp']), str(i['joined_at']), i['os']]
        row = [deli.format(x).encode(encode) for x in raw_row ]
        writer.writerow(row)
 
    return response

1. CSV匯出的時候，會遇到excel格式自動判斷錯誤的問題，後來在下面這篇看到解法：
http://stackoverflow.com/questions/165042/stop-excel-from-automatically-converting-certain-text-values-to-dates

2. 官方文件
https://docs.djangoproject.com/en/dev/howto/outputting-csv/