Laravel queue environment

Ran in to an issue running queue through supervisor. I’ll give a quick rundown of my setup, the issue and solution.

I have supervisor running the queue daemon.

[program:laravel_queue]
command=php /var/www/example.com/artisan queue:listen
autostart=true
autorestart=true
stderr_logfile=/var/log/laraqueue.err.log
stdout_logfile=/var/log/laraqueue.out.log

The key part check on is the env variable isn’t set automatically for the queue. You’ll need to set it via the —env flag.

[program:laravel_queue]
command=php /var/www/example.com/artisan queue:listen --env=prod
autostart=true
autorestart=true
stderr_logfile=/var/log/laraqueue.err.log
stdout_logfile=/var/log/laraqueue.out.log

That’s what I was missing. Once I `sudo supervisorctl restart laravel_queue` all was well with the queue.

Simple Python Web Scraper

I needed a simple html only scraper. (This doesn’t use js, won’t pull down data via AJAX). I found an example on another site, thetaranights.com, but it wasn’t exactly what I needed. It only pulled the data and printed it to screen. I added a list to loop through and auto saving by url name to a html file.

import mechanize  #pip install mechanize

br = mechanize.Browser()
br.set_handle_robots(False)
br.addheaders = [("User-agent","Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101206 Ubuntu/10.10 (maverick) Firefox/3.6.13")]

sign_in = br.open("https://this.example.com/login")  #the login url

br.select_form(nr = 0) #accessing form by their index. Since we have only one form in this example, nr =0.
#br.select_form(name = "form name") Alternatively you may use this instead of the above line if your form has name attribute available.

br["email"] = "email or username" #the key "username" is the variable that takes the username/email value

br["password"] = "password"    #the key "password" is the variable that takes the password value

logged_in = br.submit()   #submitting the login credentials

logincheck = logged_in.read()  #reading the page body that is redirected after successful login

urls = ["https://this.example.com/some/page","https://this.example.com/some/page2"]

for url in urls:
	req = br.open(url).read()
	filename = url.split('/')[-1] + ".html"
	f = open(filename, 'w')
	f.write(req)
	f.close()

Which produces 2 files:
page.html
page2.html

GIT push from one repo to another

Splitting off from a branch to another git repo seems like it would suck. In reality it’s simple. Two lines and you’re set.

Things you’ll need:

  • Source repo pulled down locally
  • Target repo created on your git server (GitHub, BitBucket, GitLab, etc)
  • Url for target git repo
  • Branches

$ cd /path/to/source
$ git remote add [email protected]:my_team/my_awesome_target_repo.git
$ git push targetrepo my_branch_to_create_off_of:master

That’ll create the master branch off of the branch “my_branch_to_create_off_of” from your local repo.

Bash test if file exists

Using bash I needed to check that a file exists, then do something. This one turns out to be super simple. I wanted to conditionally load a db file if it existed in this case.

What I needed was, a check for the file in bash. Then execute the command. In this instance it was to load a postgres restore file.

db_dump="/home/vagrant/files/db.dump"
[ -f $db_dump ] && printf "Loading DB...\n";sudo su postgres -c "pg_restore -d db_name_here $db_dump;" || printf "No db dump to load\n"

Another way to write this is to use a full if/else conditional. Admittedly this is a little cleaner.

db_dump="/home/vagrant/files/db.dump"
if [ -f "$db_dump" ]
then
	printf "Loading DB...\n";
        sudo su postgres -c "pg_restore -d db_name_here $db_dump;"
else
	printf "No DB file found.\n";
fi

How to call a python function by using a variable python

I needed to, in a programmatic way, determine the function name and then call the function.

Here’s an example use case. I have a bunch of functions called, [“function1”, “function2”, “function3”, “function4”, etc] Then I have a function that takes a number as a param. Then should return the proper function.

class my_awesome_class():
    def function1(self, number):
        # do something cool
        pass

    def function2(self, number):
        # do something cool
        pass

[...]

    def test_function(self, number, data):
        function_string = ''.join(['function',str(number)])
        try:
            t = getattr(my_awesome_class(), function_string)
            return t
        except AttributeError:
            print 'function not found "%s" (%s)' % (function_string, data)

October CMS How to replace component templates from plugins

How can you replace component templates from plugins? IE I want to replace the posts default template in the blog plugin with my own markup. Post List from the blog plugin

— Edit —

Use the override of a template instead. https://octobercms.com/docs/cms/components#overriding-partials

— Old way —

Turns out it is easy, just copy and paste in to your own partial. I typically name mine prefixed with the plug-in. “blog-plugin/post-list.htm”

Then just modify to your specs and insert in the page.

Originally:

{% component 'blogPosts' %}

Becomes:

{% partial 'blog-plugin/posts-list.htm' %}

See the comments below for other solutions. In the docs it notes that you can override the partial that is used.