Pycvf

Pycvf

• 1.
  PyCVF Overview of the Python Computer Vision Framework (preliminary version) Bertrand NOUVEL Japanese French Laboratory for Informatics bertrand.nouvel@gmail.com May 2010    
• 2.
  Traditional Framework Machine Create Compute Extract Compute Fusion Learning Video DB Shots Keyframes Features Rendering Indexing Program 1 Program 2 Program 3 Program 4 Program 5 Program 6 Program 7 ● Too many steps ● Many access to files : low performances ● Difficult to check and to run ● Not easy to perform testings    
• 3.
  Models We Needto Represent ● Learning all along the process; ● Many class­specific learners; ● Link with highlevel ontologies ● Ability to provide top­down and bottom­up  communications      Too complex fort traditional frameworks ! → Need for improvement    
• 4.
  Can We DoBetter ? ● Avoid usage of archaic file formats ● Use better software design ● Inspire from what is meant to be ”easy” Blender OpenCV     PureData
• 5.
  PyCVF is aFramework OPENCV WEKA Dependencies Libraries MPT Orange Toolkits (features) Marsyas R Framework PyCVF (uniformize concepts) Applications Analyze Mosaicing Synthetize Indexer Recognizer Textures Loop generator (use and extend the framework)    
• 6.
  Essential PyCVF Concepts DB Database Models Nodes Machine Datatypes Structures Experiment Learning   Models  
• 7.
  PyCVF Datatypes ● Images → 3d ­ Numpy Array ● Video → Video Player Object (play/pause/seek) ● Audio → Audio Player Object (play/pause/seek) ● Vectors → 1d ­ Numpy Array  ● Vectorset → 2d – Numpy Array Datatypes can solve the following questions ? ● How to store the data ? ● How to display the data ?    
• 8.
  PyCVF Databases Supported operations ● Get Datatype ● List Items  DB ● Query ● List Keys ● List Labels ● Query Label    
• 9.
  Example PyCVF Image Databases ImageDatabases Online Databases Database : Mathematical functions Cellular Automatas... Pseudo-database : device Folder-based Databases    
• 10.
  Example Meta databases ● Limit : Limit the number of elements ● Transformed : Apply a transformation  ● Exploded : Obtain sub­elements ● TrainDB : Create a training associated with a seed ● TestDB : Create a test set associated with a seed ● Randomized : Shuffle the elements of a database ● Agglomerated : Merge databases together ● As_once : Return databases as elements     
• 11.
  Nodes ● Processing units ● Dynamic typing ● Associated Storage ● Interconnectable ● Labels ● May raise Exceptions (cf. bag_of_words)    
• 12.
  Model=PyCVF Dataflow pycvf_features_view -m ”image.laplace()-image.keypoints.harris()” Laplace Harris Corner Detector    
• 13.
  PyCVF Dataflow pycvf_features_view -m ”image.laplace()-image.keypoints.harris()+image.watershed()” Laplace Harris Corner Detector WaterShed    
• 14.
  Essential Applications db_view installdb model_run evaluate_db features_view PyCVF create_index confusion_matrix distance_matrix nearest_neighbors run_experiment remote_job_server compute_features    
• 15.
  Evaluate Database pycvf_dbevaluator .... ● PyCVF include a  generic tool to help  you to label your  database ● Works with any  datatypes and any  database.    
• 16.
  Observe Features pycvf_model_features_view –db'image.kanji()' --model 'image.keypoints.sift()' ● Easy debugging ● Better intuition ● Portable    
• 17.
  Nearest Neighbors Query pycvf_neareastneighbors_qtapp–db ”image.kanji()” -m ”free('x.ravel()')” -n 5 -b25 -s kanjiidx    
• 18.
  Nearest Neighbors Query pycvf_neareastneighbors_panda–db ”image.kanji()” -m ”free('x.ravel()')” -n 5 -b25 -s kanjiidx    
• 19.
  Extract and Analyze NearestNeighbor Graphs pycvf_neareastneighbors_qtapp –db ”image.kanji()” -m ”free('x.ravel()')” -n 5 -b25 -s kanjiidx    
• 20.
  Distance Matrices pycvf_distance_matrix –db”from_trackfile('HOG')”    
• 21.
  Making Experiments ● Easy sampling of  parameter space ● Automatic  parallelization of  runs ● Specific graphics ● Specific feature ● Minimization Performances of PyCVF of index queries    
• 22.
  NEW DJANGO + PyCVF = pycvf_remote_job_server & python manage.py runserver &    
• 23.
  Browsing Databases ● On­the­fly  transformations ● Easy queries to already  computed index    
• 24.
  Viewing Element 1by 1 ● On­the­fly  transformations ● Easy queries to already  computed index    
• 25.
  Revision Plan 0.2 0.3 ● Do call for volunteers ● Add Features ● Ensure global design ● Improve feature  ● Easier install  stability ● Increase core­robustness ● Provide exhaustive set  of basic features ● Regression tests  ● Start add new features    
• 26.
  See you soon    
• 27.
  Included Wrappers ● PyFFMPEG is able to  PyFFMPEG play in realtime 121  videos at the same  time.    
• 28.
  Included Wrappers Implements  ● OpenCV-Cython exhaustively C API  ● Numpy compatibility  ● Accessible pointers ● May be deprecated due  to new C++ wrappers    
• 29.
  Included Wrappers ● Fast nearest neighbor  PySASH search ● Patented + NOT GPL    
• 30.
  Included Wrappers ● Fast nearest neighbor  PyLSH search with LSH ● Different  implementations ● Still Buggy ● Contributions are  welcome    
• 31.
  Included Wrappers PySIFT ● Numpy compatible  version of PySIFT ●