Simplified Web Development with JSON and the Twig Ternary Operator

Introduction

I’ve been looking at ways to simplify my code, make it easier to support, make it easier to use, and easier to maintain. I have found that JSON is an ideal way to store a large amount of data in databases, and Twig is an ideal templating engine to make use of JSON data.

Example JSON Representation

Here is a real example where we have a form that has a number of checkboxes that can be checked. In this case, you can check all of the checkboxes (except the no issues one).

Transport_Section

You could design your database to have a column for each checkbox, but that would be a unwise choice; since for example just this section has 6 checkboxes, and then you will need to process all 6 of those checkboxes and whether they are checked of not.

The better way to go is to store this in JSON format in the database. For the checkbox “I don’t have a driver’s license”, we could call that “License” and it can be either true or false (whether it is checked or not). For the checkbox “I will be riding with a friend to college”, we could call that “Friend”.

So I came up with the resultant format for the JSON to store in my database like so:

{
   "License": false,
   "Transit": true,
   "No_Issues": false,
   "Friend": false,
   "Car": false,
   "Other": true,
   "Other_Value": "skateboard"
}

Notice also for the “Other:” checkbox, I made it a boolean; and if it is true, there is a “Other_Value” object that stores the string value that gets entered. Essentially I use Javascript to check when the “Other:” checkbox is selected and then I will show a input field to enter the Other value, and I store that string.

Handling the JSON in the Controller

In my case, I use Symfony as my MVC framework to develop my forms and I have to first store the form booleans as a PHP array:

// Create transportation JSON.
$transport_json = array(
   "License" => $license_bool,
   "Transit" => $transit_bool,
   "No_Issues" => $trans_no_issues_bool,
   "Friend" => $friend_bool,
   "Car" => $car_bool,
   "Other" => $trans_other_bool,
   "Other_Value" => $trans_other_value
);

Then I use the standard PHP function json_encode to store the Transportation JSON data in my assessment object:

// Set the tranport JSON in the assessment.
$assess->setTransport( json_encode($transport_json) );

Also, you need to persist (uses Doctrine) the assessment object to the database:

// Save the assessment to db.
$em->persist( $assess );
$em->flush();

Then to pass the saved JSON back from the database and view in a Twig template, we use the standard PHP json_decode function to pass the JSON data as a parameter to the Twig assessment template:

return $this->render('default/viewIndNeedsAssess.html.twig', array(
        'stu' => $student,
        'assess' => $student->getIndneedForm(),
        'transport' => json_decode($student->getIndneedForm()->getTransport(), true),
        ...
));

Handling the JSON in Twig with Ternary Operator

Now finally since we can reference the “transport” JSON array in the Twig template, we can easily use the ternary operator to display a view of the completed form. In my case I wanted to show the form results in a html table within “td” elements, I wanted to show checkbox icons that show whether the form is checked or not.

Here is the Twig code I used to achieve the best results:

<td>
   <b>Transportation:</b>

    {{ transport['License'] ? '☑' : '☐' }}
      I don't have a driver's license

    {{ transport['Transit'] ? '☑' : '☐' }}
      I will be using the Kern Regional Transit / Taft Area Transit

    {{ transport['No_Issues'] ? '☑' : '☐' }}
      I have no transportation issues

    {{ transport['Friend'] ? '☑' : '☐' }}
      I will be riding with a friend to college

    {{ transport['Car'] ? '☑' : '☐' }}
      I have my own car

    {{ transport['Other'] ? '☑' : '☐' }}
      Other: {{ transport['Other_Value'] ?? '' }}</td>

Notice in the above ternary, I check if “transport[‘License’]” is true, if it’s true, then I show the code “☑” which is a checked checkbox.

And here is a screenshot with the final way the form looks when viewing a completed (submitted) form:

Transport_Final

Apache & Symfony on RHEL

Introduction

For a Symfony (an mvc framework) based project that I’m currently working, I’ve had to install Apache (2.2) and Symfony 3 (and PHP 7) on Red Hat Enterprise Linux (RHEL). This blog outlines the steps I had to go through. This may be helpful for someone in the future.

Install/Configure Apache

Installing Apache on RHEL is as simple as running the following command:

yum install httpd

You need to prepend the above command with “sudo” if you don’t have admin privileges (which is typical).

Once installed, you’ll need to edit the config file:

vi /etc/httpd/conf/httpd.conf

If you notice when using vi that comments are shown with a incredibly hard to read dark blue color, then you’ll want to edit your “~/.vimrc” file to at least add “:color desert”. You can read more about it in my vim runtime control blog article.

Edit the httpd.conf file to at least make these changes:

ServerAdmin youremail@somedomain.com
...
# Use hostname if dns can be used, otherwise use IP address.
ServerName machine.host.name:80
#ServerName 192.168.0.10:80

That’s enough config until we install Symfony. Use “:wq!” to save the changes.

Install PHP 7.0

By default, RHEL would install PHP 5.x (probably PHP 5.3 at this time), and this may not actually work for most modern PHP frameworks. PHP 7.0 has about a 2 times performance increase over PHP 5.x and it’s becoming readily adopted. Thus it’s highly recommended to install PHP 7.0 these days.

To install PHP 7.0 on RHEL, you’ll need to enable the EPEL and IUS repositories. Use the following commands (run in your home directory) to first install EPEL:

wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm
sudo rpm -Uvh epel-release-*.rpm

Then install the IUS repo:

wget https://centos6.iuscommunity.org/ius-release.rpm
sudo rpm -Uvh ius-release*.rpm

When you’re using yum to search for PHP 7.0 items to install, you need to search for “php70u”. This identifies PHP 7 verses PHP 5, which is simply “php”. Now to install PHP 7 and some of the requirements needed for Symfony (and other PHP frameworks), run the following command:

sudo yum install php70u-cli php70u-json php70u-ldap php70u-mbstring php70u-pdo php70u-mysqlnd php70u-xml php70u-process

After running the above command, run “php -v” to verify it shows PHP version 7.

Install MariaDB

You should probably now be using MariaDB instead of MySQL. If you’re not sure why MariaDB is preferable, read the MariaDB article on Wikipedia. On RHEL, you’ll need to add a repository file. First go to: https://downloads.mariadb.org/mariadb/repositories/ and follow the instructions by selecting a Linux Distro and selecting the latest stable version of MariaDB.

Copy the text generated and paste it into the repo file that you will create with vi:

sudo vi /etc/yum.repos.d/MariaDB.repo

Save the file in vi (“:wq!”), then run the following command:

yum install MariaDB-server MariaDB-client

After install, start the service:

sudo service mysql start

Verify it starts correctly. If it doesn’t, try to figure out why before proceeding. Then change the root password to something more secure (especially if this is going to be a production machine). Use the following command, where newpassword is what you want to change the new password to:

sudo mysqladmin -u root password newpassword

Install Composer

Composer is a dependency management software for PHP, and is needed for Symfony and many other PHP frameworks. Run the following commands:

cd ~
php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
php -r "if (hash_file('SHA384', 'composer-setup.php') === '92102166af5abdb03f49ce52a40591073a7b859a86e8ff13338cf7db58a19f7844fbc0bb79b2773bf30791e935dbd938') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;"
php composer-setup.php
php -r "unlink('composer-setup.php');"composer

This creates a “composer.phar” in your home directory. You need to copy it to the “/usr/local/bin” folder to make it globally available:

sudo mv composer.phar /usr/local/bin/composer

Run the command “composer” to verify it works.

Install Symfony Binary

Now install the latest Symfony binary. Don’t use the Symfony available with yum, since that is typically really old. Change to your home directory and run the following:

cd ~
sudo curl -LsS https://symfony.com/installer -o /usr/local/bin/symfony
sudo chmod a+x /usr/local/bin/symfony

Now, change to your Apache root web folder:

cd /var/www/html

Then let’s create a new Symfony project. This is a sample project that you can try out, and get to know how Symfony works:

symfony new testproject

Note: If for some reason “symfony” doesn’t run, you may need to run it from the absolute path or run with “sudo” or even using both. i.e. “sudo /usr/local/bin/symfony new testproject”.

The above creates the folder “testproject” under “/var/www/html”. Normally, you run the above commands as a system user not as the “apache” user, so you need to change the user:group of the folder and files. Do this recursively:

sudo chown -R apache:apache testproject

Also set some special permission on the var folder:

sudo setfacl -R -m u:"apache":rwX -m u:apache:rwX var
sudo setfacl -dR -m u:"apache":rwX -m u:apache:rwX var

Let’s go back to update the Apache config.

Apache Config

We need to update the DocumentRoot and Directory directives to the Symfony web folder we created above. The web folder will be under testproject, i.e. “testproject/web”. So make the following changes to the Apache config:

sudo vi /etc/httpd/conf/httpd.conf

DocumentRoot "/var/www/html/testproject/web"
...
<Directory "/var/www/html/testproject/web">
...
    Options Indexes FollowSymLinks MultiViews
...
    AllowOverride All
...
</Directory>

The above changes are needed also for mod_rewrite. Start the Apache server:

sudo service httpd start

Make sure it starts with “[OK]” normally, if not check the logs in “/var/log/httpd/”.

Verify Symfony Requirements

There were a large number of configuration items just performed, so it’s probably a good idea to check the Symfony requirements to verify everything you’ve installed is correct and no further changes are needed. There is a “symfony_requirements” script in the bin folder to achieve this specific purpose. From the “/var/www/html/testproject” folder run:

php bin/symfony_requirements

You should get a green [OK] if everything is fine, otherwise it prints out anything that you need to address.

RHEL SELINUX & IPTABLES

Most likely if you installed Apache fresh, then SELINX won’t be configured for the web folders, and quite possibly iptables are configured. Run the command “sudo iptables -S” to check the iptables configuration. If there is anything other than ACCEPT, and nothing specific to port 80, then most likely this will block your web port. Run “sudo iptables -F” to flush the tables.

The easiest way to quickly disable SELINUX is using “setenforce permissive”.

I’m not giving guidelines here on how to setup iptables or SELINUX, as for a production environment, a lot of thought should be put into correct configuration, and I only intend to show how to use Symfony here.

Checking Everything

Presuming your Apache server is running (run a “sudo service restart” if it is not), you should be able to use wget to check if everything is working “locally”:

cd ~
wget 192.168.0.10

This should copy an “index.html” to your home folder. Use “vi index.html” to check the contents. If you get errors, you need to investigate the problems.

After checking locally, use another machine and enter the hostname or IP address in the URL field of a browser. You should get a Symfony Welcome! message.

 

CakePHP Bookmarks Tutorial

Introduction

I have been going through the CakePHP Bookmarker Tutorial, and have struggled with getting it to work following the instructions outlined on those pages. In this blog, i will outline what steps I need to follow in order to get this to work.

Environment

My environment is as follows:

  • CentOS 6 virtual machine (on VirtualBox)
  • PHP 7.0 (php70u) with Apache 2.2.15 with mod_rewrite

To install CakePHP you will need Composer and also ext-intl. On CentOS, ext-intl is installed using `yum install php70u-intl`.

Installing

To install the project, run this command:

composer self-update && composer create-project --prefer-dist cakephp/app bookmarks

Notice I’ve used a project name of “bookmarks” instead of “bookmarker”.

Creating the Database

To create the database you’ll need to login to MySQL (presumes MySQL is used) and then create the specific database used. Let’s call it “cakedb”. Here is the sequence of commands:

mysql -u username -p
Enter password: userpassword
MariaDB [(none)]> CREATE DATABASE cakedb;
MariaDB [(none)]> USE cakedb;
MariaDB [cakedb]> ALTER DATABASE cakedb CHARACTER SET utf8;

In the above commands, “username” is the MySQL username that can login and create databases. I had to set the character set to UTF-8 so that when creating the tags table, the length of “title VARCHAR(255)” won’t throw an error (ERROR 1071) about the maximum key length.

Run these SQL commands to create the database tables:

CREATE TABLE users (
    id INT AUTO_INCREMENT PRIMARY KEY,
    email VARCHAR(255) NOT NULL,
    password VARCHAR(255) NOT NULL,
    created DATETIME,
    modified DATETIME
);

CREATE TABLE bookmarks (
    id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
    user_id INT NOT NULL,
    title VARCHAR(50),
    description TEXT,
    url TEXT,
    created DATETIME,
    modified DATETIME,
    FOREIGN KEY user_key (user_id) REFERENCES users(id)
);

CREATE TABLE tags (
    id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
    title VARCHAR(255),
    created DATETIME,
    modified DATETIME,
    UNIQUE KEY (title)
);

CREATE TABLE bookmarks_tags (
    bookmark_id INT NOT NULL,
    tag_id INT NOT NULL,
    PRIMARY KEY (bookmark_id, tag_id),
    FOREIGN KEY tag_key(tag_id) REFERENCES tags(id),
    FOREIGN KEY bookmark_key (bookmark_id) REFERENCES bookmarks(id)
);

After creating the tables, exit MySQL:

MariaDB [cakedb]> exit;

Modify Config File

Open the file “config/app.php” and set the database username and password, and database name:

    'Datasources' => [
        'default' => [
            'className' => 'Cake\Database\Connection',
            'driver' => 'Cake\Database\Driver\Mysql',
            'persistent' => false,
            'host' => 'localhost',
            /**
             * CakePHP will use the default DB port based on the driver selected
             * MySQL on MAMP uses port 8889, MAMP users will want to uncomment
             * the following line and set the port accordingly
             */
            //'port' => 'non_standard_port_number',
            'username' => 'username',
            'password' => 'userpassword',
            'database' => 'cakedb',
            'encoding' => 'utf8',
            'timezone' => 'UTC',
            'flags' => [],
            'cacheMetadata' => true,
            'log' => false,

Generate the Scaffolding Code

From the “bookmarks” directory, run the following commands to create the scaffolding code needed for the application:

bin/cake bake all users
bin/cake bake all bookmarks
bin/cake bake all tags

Once this the above is done, you should be able to start the server. For my setup, the vm I’m using is on a different IP address than my locahost, so I needed to specify a specific IP address and port. This is the command that I ran:

bin/cake -H 192.168.0.100:88

Then in a web browser, enter the URL ‘http://192.168.0.100:88/bookmarks&#8217;.

 

 

Symfony AD Integration

Introduction

Recently, I’ve been investigating Symfony as a solution for a student petitions web application that I’m working on at Taft College. Symfony is a PHP framework that greatly simplifies the creation of web applications. It’s essentially at Model View Controller (MVC) framework.
This blog outlines the enabling of Active Directory integration with Symfony. My reason for documenting it here, was because when I tried searching online for information/examples on how to achieve this, there were references on how to install various components but there were no real step-by-step instructions on how to verify everything it actually working. Hopefully this will help someone else out who is struggling to set this up properly.

Installing Symfony

This blog presumes you have some Linux OS and are using LAMP (Linux Apache MySql/MariaDB PHP) and each of those is configured correctly. You will need to install Symfony and create your Symfony project that you’ll be using to setup the Active Directory integeration. The Symfony Book Installing and Configuring Symfony gives instructions on how to install Symfony (binary) and Composer (you’ll need this too); I recommend to fully go over most of the Symfony Book, as it will help you get familiar with Symfony. Once you’ve created your Symfony project, you can proceed with the next steps.

Install FOSUserBundle

The Friends Of Symfony User Bundle (FOSUserBundle), is a bundle that persists and fetches users in a database. To install it, use composer with the following command within the root of your Symfony web application. For example, let’s presume your apache DocumentRoot is ‘/var/www/html’ and you’ve created your Symfony project application as ‘test’ in the DocumentRoot, then you would run this command under ‘/var/www/html/test’. Run the following command to install FOSUserBundle:

composer require friendsofsymfony/user-bundle “~2.0@dev”

Install PHP LDAP

In this example, I’m using Centos 6. So if you have a different Linux distro, you may have to install with different commands (i.e. rpm, apt-get). Run the following command to install ext-ldap:

yum install php-ldap

There is no post install configuration required.

Install FR3DLdapBundle

The FR3DLdapBundle provide LDAP authentication. To install it, run the following composer command from your Symfony project root:

composer require fr3d/ldap-bundle “3.0.*@dev”

Configuration

You will need to edit a number of files described here to setup the user, security and Active Directory configuration. First, open the file ‘app/AppKernel.php’ and add the FOSUserBundle and FR3DLdapBundle bundles to the registerBundles function:

<?php
 // app/AppKernel.php

public function registerBundles()
 {
   $bundles = [
      // ...
      new FOS\UserBundle\FOSUserBundle(),
      new FR3D\LdapBundle\FR3DLdapBundle(),
      // ...
   ];
 }

Create a file called ‘FOSUser.php’ under ‘src/AppBundle/Entity’. Note: create the Entity directory if it doesn’t exist. Add the following contents to FOSUser.php:

<?php

// src/AppBundle/Entity/FOSUser.php
 namespace AppBundle\Entity;

use FOS\UserBundle\Model\User as BaseUser;
 use FR3D\LdapBundle\Model\LdapUserInterface;
 use Doctrine\ORM\Mapping as ORM;

/**
   * @ORM\Entity
   * @ORM\Table(name="fos_user")
   */
 class FOSUser extends BaseUser implements LdapUserInterface
 {
 /**
   * @ORM\Id
   * @ORM\Column(type="integer")
   * @ORM\GeneratedValue(strategy="AUTO")
   */
 protected $id;

/**
   * @ORM\Column(type="string")
   */
 protected $dn;

public function __construct()
 {
   parent::__construct();
   if (empty($this->roles)) {
      $this->roles[] = 'ROLE_USER';
   }
 }

/**
   * {@inheritDoc}
   */
 public function setDn($dn)
 { $this->dn = $dn; }

/**
   * {@inheritDoc}
   */
 public function getDn()
 { return $this->dn; }
 }

Open the file ‘app/config/routing.yml’ and make the following change:

#    app/config/routing.yml
 ...
 fos_user:
   resource: "@FOSUserBundle/Resources/config/routing/all.xml"

The above setting enables Symfony routing based on configuration from the file vendor/friendsofsymfony/user-bundle/Resources/config/routing/all.xml which enables the routes such as “/login” and “/logout”, providing standard login/logout capability. The Bundle can be overriden but that is not discussed here.

Open the file ‘app/config/security.yml’ and make the following changes. NOTE: This is only my example; Read the Symfony Book > Security section for other examples on how you might configure security for your own scenario. Symfony is highly configurable, and this example is only used as a guideline on helping you to understand how security works in Symfony.

#    'app/config/security.yml'
 # To get started with security, check out the documentation:
 # http://symfony.com/doc/current/book/security.html
 security:
 # Preserve plain text password in token for refresh the user.
 # Analyze the security considerations before turn off this setting.
    erase_credentials: false

encoders:
  FOS\UserBundle\Model\UserInterface: bcrypt

role_hierarchy:
  ROLE_ADMIN:       ROLE_USER
  ROLE_SUPER_ADMIN: ROLE_ADMIN

providers:
  chain_provider:
     chain:
        providers: [fos_userbundle, fr3d_ldapbundle]
  fr3d_ldapbundle:
     id: fr3d_ldap.security.user.provider
  fos_userbundle:
     id: fos_user.user_provider.username

firewalls:
 # disables authentication for assets and the profiler, adapt it according to your needs
 dev:
   pattern: ^/(_(profiler|wdt)|css|images|js)/
   fr3d_ldap:  ~
   security: false
   form_login:
     always_use_default_target_path: true
     default_target_path: /profile
   logout:       true
   anonymous:    true

 main:
   pattern: ^/
   fr3d_ldap:  ~
   form_login:
     always_use_default_target_path: true
     default_target_path: /profile
   logout:       true
   anonymous:    true

 access_control:
   - { path: ^/login$, role: IS_AUTHENTICATED_ANONYMOUSLY }
   - { path: ^/register, role: IS_AUTHENTICATED_ANONYMOUSLY }
   - { path: ^/resetting, role: IS_AUTHENTICATED_ANONYMOUSLY }
   - { path: ^/admin/, role: ROLE_ADMIN }

 

Open the file ‘app/config/config.yml’ and add the following lines at the bottom of the file:

#    app/config/config.yml
 ...
 # Friends of Symfony user
 fos_user:
   db_driver: orm
   firewall_name: main
   user_class: AppBundle\Entity\FOSUser

fr3d_ldap:
   driver:
      host: ldap.forumsys.com
   user:
      baseDn: dc=example, dc=com
      attributes:
         - { ldap_attr: uid,  user_method: setUsername }
         - { ldap_attr: mail,  user_method: setEmail }
      filter: (&(ObjectClass=person))

In the above configuration, I’m using the online test LDAP server ‘ldap.forumsys.com’, which is available for anyone to use to test LDAP authentication. Information on this LDAP test server is found at the following web site: http://www.forumsys.com/tutorials/integration-how-to/ldap/online-ldap-test-server/

Update Schema

Run the following command from the project root directory:

php bin/console doctrine:schema:update –force

This updates the database schema.

Testing Operation

This section describes the steps needed to verify that the above configuration is actually working and doing what it is supposed to. The key to troubleshooting is checking the ‘var/logs/dev.log’ entries. Since the login page configured is ‘^/login$’, you need to point your browser to your apache Directory path. I’m presuming here that you’ve configured this as your Symfony root/web path (the web folder under your Symfony project root directory). Then in your browser enter the following URL:

http://host.name/app_dev.php/login

Where ‘host.name’ is either an IP address or hostname where your apache web server is running. Notice the path contains ‘app_dev.php’. This is a file located in the web folder. If you are not running your browser on the same host that the apache web server is running on, then you’ll need to open this file and add the IP address of your host machine to the ‘REMOTE_ADDR’ array. NOTE: Only add trusted hosts to the ‘REMOTE_ADDR’ array. The following line shows an example where my host machine has an IP address of ‘192.168.40.208’:

//    web/app_dev.php
 ...
 if (isset($_SERVER['HTTP_CLIENT_IP'])
   || isset($_SERVER['HTTP_X_FORWARDED_FOR'])
   || !(in_array(@$_SERVER['REMOTE_ADDR'], ['192.168.40.208', '127.0.0.1',
   'fe80::1', '::1']) || php_sapi_name() === 'cli-server')
 )

After you browse to the login page, you should should see labels and text fields for entering username and password and a submit button. Enter a username of “riemann” and password of “password”, then click “security.login.submit”, then you should the profile information for the user. Click the “layout.logout” link to logout.

If the login/logout works correctly, then the normal login URL to use is:

http://host.name/login

And to logout, simply use:

http://host.name/logout

 

Getting AD Authentication to Work

Once you have verified the above (verifies LDAP authentication works), then you need to change your “config.yml” to use your Active Directory server. Edit your “config.yml” as follows (the following data is obsfucated):

# FILE: app/config/config.yml
 ...
 fr3d_ldap:
   driver:
      host: my_ad_server.hostname.com
      username: ad_admin_user_account
      password: admin_password
      accountDomainName: DOM.hostname.com
      accountDomainNameShort: DOM # i.e. prefix you use to login.
   user:
      usernameAttribute: sAMAccountName
      baseDn: cn=Users,dc=hostname,dc=com # Use a LDAP browser to verify this.
      filter: (&(ObjectClass=Person))
      attributes:
         - { ldap_attr: samaccountname, user_method: setUsername }
         - { ldap_attr: mail, user_method: setEmail }

You should now be able to login to your Symfony system using your Active Directory accounts. Always start in the dev environment (append ‘app_dev.php’ to URL) to check first that things are working normally, and check the var/logs/dev.log for error messages.