Newer
Older
Import / applications / HighwayDash / ports / Framework / meta-tool.py
#!/usr/bin/python
## @package docstring
#
# @page Other
# @subpage MetaTool
# @page MetaTool Meta Tool
# 
# ### Documentation for meta-tool
# 
# Meta-tool allows you to embed commands inside the source code in comments.
# The meta-tool can read these and use them to do various tasks like code generation.
# It is a multi-function tool, so is not specific for particular tasks, if you
# use your imagination, it can be very powerful.
#  
# It can be used as a driver for running unit-tests for example. Or can be used
# to convert binary files in to includable resources with the code.
# 
# Because it can also be used to invoke the compiler, it could be used to build a
# buildsystem. Or as another way to handle meta-programming but in a more familiar
# language and in a consistent manner instead of how template-meta-programming feels
# like you are using a totally different langauge.
#
# The idea being that the build steps first invoke the meta-tool which could compile
# sections of the code (with differently defined #ifdef blocks) which are used to
# create a program to run which can generate or output c++ code that can then be
# included to be be compiled in the 2nd compile phase to create the final target.
#
# ### Example Usages:
# 
#  - ./meta-tool.py --gen ./ResourceLoader.cpp
#  - ./meta-tool.py --tests ./ResourceLoader.cpp
#
# ### How To Use:
#
# MetaData blocks are added to C++ code in a comment for the meta-tool to read.
# An example meta-data block could look like this:
#
# ```
# /*************************** MetaData ****************************
# # See: meta-tool.py
# # to test, compile with:
# UNIT_TEST_CMD="g++ -D UNIT_TESTS --std=c++14 -I. -pthread -o unittest ResourceLoader.cpp Utilities.cpp Common.cpp"
# CODE_GEN_CMD="g++ -D CODE_GEN --std=c++14 -I. -pthread -o codegen ResourceLoader.cpp Utilities.cpp Common.cpp"
# **************************** MetaData ***************************/
# #include "Utilities.h"
# #ifdef CODE_GEN
# void code_gen() { Utilities::xxd("gen.h", "Font6.png", "Font6"); }
# #else
# #include "gen.h"
# #endif
# ```
#
# It finds a comment block starting with '** MetaData **' and ending with a line with '** MetaData **'.
# Note that lines beginning with a hash ('#') are ignored. All other lines are evaluated.
#
# If the command line parameter of '--gen' is added, it will evaluate the CODE_GEN_CMD, and then run ./codegen
#
# If the command line parameter of '--tests' is added, it will evaluate the UNIT_TEST_CMD, and the run ./unittest
#
# You will notice in the example, the CODE_GEN_CMD adds the compiler options "-D CODE_GEN". This sets this define.
# The code that will get compiled will be the "void code_gen() { ... }" function. This calls the xxd function
# which is similar to the Linux xxd command. It reads in the file "Font6.png" and makes it in to a header file
# called "gen.h" with the contents of the PNG image in a variable named "Font6".
#
# On the other side of the #if CODE_GEN is an include statement of the generated file.
#
# So what this does is, if run in code-generation mode, will make the gen.h file that is needed out of the Font6.png.
# Otherwise if compiled normally, it is assumed the code generation has already been done and includes the generated
# gen.h.
#
# ### Run
#
# If a file called "meta-tool-test.h" has this contents:
#
# ```
# /*************************** MetaData ****************************
# os.system('./meta-tool.py meta-tool-test.h --dump > tmptmp.h')
# os.system('echo "#pragma once" >> tmptmp.h')
# os.system('echo >> tmptmp.h')
# os.system('xxd -i Font6.png >> tmptmp.h')
# os.system('mv tmptmp.h meta-tool-test.h')
# **************************** MetaData ***************************/
# ```
#
# Then invoking meta-tool with "--run" like this:
#
# ```
# ./meta-tool.py meta-tool-test.h --run
# ```
#
# It will execute the python commands given in the file. What they do is to extract the meta-data from the file
# and output it to a new file and then append to this new file a converted PNG as data in the file. This new file
# is then copied over the original. Invoking this command again will essentially re-fresh the data conents of the
# file.
#
# ### Notes
#
# The meta-tool in a limited way lets programmers add custom command line commands which the code might need to be
# run to generate code. But this is not good from a cross-platform point of view, as Linux commands might not work
# well on other systems and there could be minor incompatibilities.
#
# ### Future Work
#
# What probably makes sense is to expose adding python script which is interpreted in to the meta-data. Then it
# will work cross-platform and be a bit more controlled. Also certain functions could be available to call in this
# python to do common tasks such as invoking the compiler. This could allow this function to map to the appropriate
# commands for the given type of platform or add extra command line options that might be required.
#

import argparse
import os

parser = argparse.ArgumentParser(description='Generic meta compiler tool')
parser.add_argument('file', type=str, help='a file to process')
parser.add_argument('--dump', action='store_true', help='print the meta commands from the file')
parser.add_argument('--tests', action='store_true', help='run unit tests using meta commands from the file')
parser.add_argument('--gen', action='store_true', help='run code generation using meta commands from the file')
parser.add_argument('--coverage', action='store_true', help='run code coverage tests using meta commands from the file')
parser.add_argument('--profile', action='store_true', help='profile code using meta commands from the file')
parser.add_argument('--run', action='store_true', help='run the meta commands from the file')
args = parser.parse_args()

# Example usage:
#   Compile(['one.cpp', 'two.cpp'], 'target')
#   Compile(['one.cpp', 'two.cpp'], 'target', ['ONE', 'TWO'], ['.'])
#   Compile(['one.cpp', 'two.cpp'], 'target', ['ONE', 'TWO'], ['.'], ['liba', 'libb'], ['/liba/lib', '/usr/lib'], '--stdc++=14')
def Compile(inputFiles, target='a.out', defines=[], includeDirs=[], libs=[], libDirs=[], options=''):
  # TODO: make this configurable / platform specific
  os.system('g++ -o ' + target
              + ''.join([' -D ' + d for d in defines])
              + ''.join([' -I'  + i for i in includeDirs])
              + ''.join([' -L'  + l for l in libDirs])
              + ''.join([' -l'  + l for l in libs])
              + ' ' + options + ' ' + ' '.join(inputFiles))

# Very similar to the xxd command-line command
def xxd(file_path, array_name, out):
  with open(file_path, 'r') as f:
    output = "unsigned char %s[] = {" % array_name
    length = 0
    while True:
      buf = f.read(12)
      if not buf:
        output = output[:-1]
        break
      else:
        output += "\n "
      for i in buf:
        output += " 0x%02x," % ord(i)
        length += 1
    output += "\n};\n"
    output += "unsigned int %s_len = %d;\n" % (array_name, length)
    out.write(output)

# Retrieves the meta-data commands from the file
def extractMeta(file):
  insideMeta=False
  metaFirstLine=True
  output=''
  for line in open(file, "r"):
    line = line.rstrip('\n')
    if '** MetaData **' in line:
      if line.endswith('*/'):
        insideMeta=False
      elif line.startswith('/*'):
        insideMeta=True
      else:
        if insideMeta:
          output=output + '\n' + line
    else:
      if insideMeta:
        if metaFirstLine:
          metaFirstLine = False
          output=output + line
        else:
          output=output + '\n' + line
  return output

# Example usage:
#   /*************************** MetaData ****************************
#   XXD('Font6.png')
#   **************************** MetaData ***************************/
# This will then replace the reset of the file with an xxd version of
# the file. It will refresh it if run "./meta-tool.py --run file.h" again.
def XXD(infile):
  with open('tmptmp.h', 'w') as out:
    out.write("/*************************** MetaData ****************************\n")
    out.write(extractMeta(args.file) + "\n")
    out.write("**************************** MetaData ***************************/\n")
    out.write("#pragma once\n\n")
    xxd(infile, infile.replace('.','_'), out) 
    os.rename('tmptmp.h', args.file)

if args.dump:
  print "/*************************** MetaData ****************************"
  print extractMeta(args.file)
  print "**************************** MetaData ***************************/"

if args.tests:
  print "unit testing"
  exec(extractMeta(args.file))
  print str(UNIT_TEST_CMD)
  os.system(UNIT_TEST_CMD)
  os.system('./unittest')

if args.gen:
  print "code generation"
  exec(extractMeta(args.file))
  print str(CODE_GEN_CMD)
  os.system(CODE_GEN_CMD)
  os.system('./codegen')

if args.run:
  print "running meta commands"
  exec(extractMeta(args.file))