Upload 60 files

.gitattributes

@@ -33,3 +33,11 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+data/1739757991209.png filter=lfs diff=lfs merge=lfs -text
+figs/DexiNed_banner.png filter=lfs diff=lfs merge=lfs -text
+legacy/figs/DexiNed_banner.png filter=lfs diff=lfs merge=lfs -text
+result/BIPED2CLASSIC/avg/1739757991209.png filter=lfs diff=lfs merge=lfs -text
+result/BIPED2CLASSIC/avg/Hyundai_Komponen_Mobil_utamaaa.png filter=lfs diff=lfs merge=lfs -text
+result/BIPED2CLASSIC/avg/jalan-rusak-di-ruas-jalan-solo-semarang-boyolali_169.png filter=lfs diff=lfs merge=lfs -text
+result/BIPED2CLASSIC/avg/TC[[:space:]]10[[:space:]]MARET[[:space:]]-[[:space:]]TIPS[[:space:]]MENGHINDAR[[:space:]]DARI[[:space:]]LUBANG[[:space:]]JALAN[[:space:]](UMUM).png filter=lfs diff=lfs merge=lfs -text
+result/BIPED2CLASSIC/fused/1739757991209.png filter=lfs diff=lfs merge=lfs -text

=1.4

@@ -0,0 +1,52 @@
+Collecting torch
+  Downloading torch-2.7.0-cp312-cp312-manylinux_2_28_x86_64.whl.metadata (29 kB)
+Collecting filelock (from torch)
+  Downloading filelock-3.18.0-py3-none-any.whl.metadata (2.9 kB)
+Collecting typing-extensions>=4.10.0 (from torch)
+  Downloading typing_extensions-4.13.2-py3-none-any.whl.metadata (3.0 kB)
+Collecting setuptools (from torch)
+  Downloading setuptools-80.8.0-py3-none-any.whl.metadata (6.6 kB)
+Collecting sympy>=1.13.3 (from torch)
+  Downloading sympy-1.14.0-py3-none-any.whl.metadata (12 kB)
+Collecting networkx (from torch)
+  Downloading networkx-3.4.2-py3-none-any.whl.metadata (6.3 kB)
+Collecting jinja2 (from torch)
+  Downloading jinja2-3.1.6-py3-none-any.whl.metadata (2.9 kB)
+Collecting fsspec (from torch)
+  Downloading fsspec-2025.5.0-py3-none-any.whl.metadata (11 kB)
+Collecting nvidia-cuda-nvrtc-cu12==12.6.77 (from torch)
+  Downloading nvidia_cuda_nvrtc_cu12-12.6.77-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
+Collecting nvidia-cuda-runtime-cu12==12.6.77 (from torch)
+  Downloading nvidia_cuda_runtime_cu12-12.6.77-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.5 kB)
+Collecting nvidia-cuda-cupti-cu12==12.6.80 (from torch)
+  Downloading nvidia_cuda_cupti_cu12-12.6.80-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.6 kB)
+Collecting nvidia-cudnn-cu12==9.5.1.17 (from torch)
+  Downloading nvidia_cudnn_cu12-9.5.1.17-py3-none-manylinux_2_28_x86_64.whl.metadata (1.6 kB)
+Collecting nvidia-cublas-cu12==12.6.4.1 (from torch)
+  Downloading nvidia_cublas_cu12-12.6.4.1-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.5 kB)
+Collecting nvidia-cufft-cu12==11.3.0.4 (from torch)
+  Downloading nvidia_cufft_cu12-11.3.0.4-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.5 kB)
+Collecting nvidia-curand-cu12==10.3.7.77 (from torch)
+  Downloading nvidia_curand_cu12-10.3.7.77-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.5 kB)
+Collecting nvidia-cusolver-cu12==11.7.1.2 (from torch)
+  Downloading nvidia_cusolver_cu12-11.7.1.2-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.6 kB)
+Collecting nvidia-cusparse-cu12==12.5.4.2 (from torch)
+  Downloading nvidia_cusparse_cu12-12.5.4.2-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.6 kB)
+Collecting nvidia-cusparselt-cu12==0.6.3 (from torch)
+  Downloading nvidia_cusparselt_cu12-0.6.3-py3-none-manylinux2014_x86_64.whl.metadata (6.8 kB)
+Collecting nvidia-nccl-cu12==2.26.2 (from torch)
+  Downloading nvidia_nccl_cu12-2.26.2-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (2.0 kB)
+Collecting nvidia-nvtx-cu12==12.6.77 (from torch)
+  Downloading nvidia_nvtx_cu12-12.6.77-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.6 kB)
+Collecting nvidia-nvjitlink-cu12==12.6.85 (from torch)
+  Downloading nvidia_nvjitlink_cu12-12.6.85-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.5 kB)
+Collecting nvidia-cufile-cu12==1.11.1.6 (from torch)
+  Downloading nvidia_cufile_cu12-1.11.1.6-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.5 kB)
+Collecting triton==3.3.0 (from torch)
+  Downloading triton-3.3.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (1.5 kB)
+Collecting mpmath<1.4,>=1.1.0 (from sympy>=1.13.3->torch)
+  Downloading mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB)
+Collecting MarkupSafe>=2.0 (from jinja2->torch)
+  Downloading MarkupSafe-3.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.0 kB)
+Downloading torch-2.7.0-cp312-cp312-manylinux_2_28_x86_64.whl (865.0 MB)
+   ━━━━━━━━━━━━━━━━━━━━━╸                   466.4/865.0 MB 2.4 MB/s eta 0:02:48

DexiNed-TF2/README.md

@@ -0,0 +1,47 @@
+In cosntruction
+
+# DexiNed-TF2
+
+
+This repo is an official unofficial version of DexiNed in TensorFlow Keras. The first official version of [DexiNed is in Tensorflow1.13](https://github.com/xavysp/DexiNed).
+
+## Requerements
+
+* Python 3.7
+* Tensorflow 2.2.
+* OpenCV
+
+## Training and Testing  Settings
+
+Either for training or testing you should set dataset_manager.py, there you could find details of datasets like size, source, and so on, you should set before running DexiNed-TF2.
+Then you can set the following code in main.py:
+```
+DATASET_NAME= ['BIPED','BSDS','BSDS300','CID','DCD','MULTICUE',
+                'PASCAL','NYUD','CLASSIC'] # 8
+TEST_DATA = DATASET_NAME[1]
+TRAIN_DATA = DATASET_NAME[0]
+...
+parser.add_argument("--model_state",default='test', choices=["train", "test", "export"])
+
+```
+Model_state should be "train" for training :)
+To train DexiNed-TF2 is similar to training in Tensorflow. For more details see [DexiNed](https://github.com/xavysp/DexiNed/blob/master/README.md).
+
+To summarize: firstly you should download and unzip the BIPED dataset hosted in [Kaggle](https://www.kaggle.com/xavysp/biped). Secondly, augment the dataset with [this ripo](https://github.com/xavysp/MBIPED). Once the BIPED is augmented run.
+
+If you want to use just for testing with single images please choice "Classic" dataset, make a dir "data" into DexiNed-TF2, and leave the images for testing into "data" dir and run in "test" mode. You will find the model's weights [Here](https://drive.google.com/file/d/19Gwa6egqzNolvX4eUoXn-SjRKzxB68AA/view?usp=sharing)
+ 
+ # Citation
+Please cite our paper if you find helpful,
+```
+@InProceedings{soria2020dexined,
+    title={Dense Extreme Inception Network: Towards a Robust CNN Model for Edge Detection},
+    author={Xavier Soria and Edgar Riba and Angel Sappa},
+    booktitle={The IEEE Winter Conference on Applications of Computer Vision (WACV '20)},
+    year={2020}
+}
+```
+```diff
++ If you find some typos or you think we can improve the code, we will appreciate your contribution
+
+```

DexiNed-TF2/dataset_manager.py

@@ -0,0 +1,280 @@
+import tensorflow as tf
+import numpy as np
+import h5py, os
+import random
+import cv2 as cv
+
+
+AUTOTUNE = tf.data.experimental.AUTOTUNE
+BUFFER_SIZE = 1024
+
+img_shape =None
+
+class DataLoader(tf.keras.utils.Sequence):
+
+    def __init__(self,data_name,arg=None, is_val=False):
+
+        self.is_training = True if arg.model_state.lower() == 'train' else False
+        self.dim_w = arg.image_width if self.is_training else arg.test_img_width
+        self.dim_h = arg.image_height if self.is_training else arg.test_img_height
+        self.args = arg
+        self.base_dir = arg.train_dir if arg.model_state.lower()=='train' else arg.test_dir
+        self.is_val = is_val
+        self.data_name =data_name
+        self.bs = arg.batch_size if self.is_training else arg.test_bs
+        self.shuffle=self.is_training
+        if not self.is_training and arg.model_state=="test":
+            i_width =  self.dim_w if  self.dim_w%16==0 else (self.dim_w//16+1)*16
+            i_height= self.dim_h if self.dim_h%16==0 else (self.dim_h//16+1)*16
+            self.input_shape = (None,i_height, i_width,3)
+            self.dim_w = i_width
+            self.dim_h = i_height
+            self.imgs_shape = []
+            # OMSIV real size= 320,580,3
+        self.data_list = self._build_index()
+        self.on_epoch_end()
+
+
+    def _build_index(self):
+
+        # base_dir = os.path.join(self.base_dir, self.args.model_state.lower())
+        list_name= self.args.train_list if self.is_training else self.args.test_list
+
+        if not self.data_name.lower()=='classic':
+            file_path = os.path.join(self.base_dir, list_name)
+            with open(file_path,'r') as f:
+                file_list = f.readlines()
+            file_list = [line.strip() for line in file_list] # to clean the '\n'
+            file_list = [line.split(' ') for line in file_list] # separate paths
+        if self.data_name.lower() in ['biped','mbiped']:
+            m_mode = 'train' if self.is_training else 'test'
+            input_path = [os.path.join(
+                self.base_dir,'imgs',m_mode,line[0]) for line in file_list]
+            gt_path = [os.path.join(
+                self.base_dir,'edge_maps',m_mode,line[1]) for line in file_list]
+        elif self.data_name.lower()=='classic':
+            file_list = os.listdir(self.base_dir)
+            input_path = [os.path.join(self.base_dir,line) for line in file_list]
+            gt_path = None
+        else:
+            input_path = [os.path.join(self.base_dir, line[0]) for line in file_list]
+            gt_path = [os.path.join(self.base_dir, line[1]) for line in file_list]
+
+        # split training and validation, val=10%
+        if self.is_training and self.is_val:
+            input_path = input_path[int(0.9 * len(input_path)):]
+            gt_path = gt_path[int(0.9 * len(gt_path)):]
+        elif self.is_training:
+            input_path = input_path[:int(0.9 * len(input_path))]
+            gt_path = gt_path[:int(0.9 * len(gt_path))]
+
+        if not self.is_training:
+            self.imgs_name = [os.path.basename(k) for k in input_path]
+            for tmp_path in input_path:
+                tmp_i = cv.imread(tmp_path)
+                tmp_shape = tmp_i.shape[:2]
+                self.imgs_shape.append(tmp_shape)
+        sample_indeces= [input_path, gt_path]
+        return sample_indeces
+
+    def on_epoch_end(self):
+        self.indices = np.arange(len(self.data_list[0]))
+        if self.shuffle:
+            np.random.shuffle(self.indices)
+
+    def __len__(self):
+        return len(self.indices)//self.bs
+
+
+    def __getitem__(self, index):
+
+        indices = self.indices[index*self.bs:(index+1)*self.bs]
+        if not self.data_name.lower()=='classic':
+
+            x_list,y_list = self.data_list
+            tmp_x_path = [x_list[k] for k in indices]
+            tmp_y_path = [y_list[k] for k in indices]
+
+            x,y = self.__data_generation(tmp_x_path,tmp_y_path)
+        else:
+            x_list, _ = self.data_list
+            tmp_x_path = [x_list[k] for k in indices]
+            x, y = self.__data_generation(tmp_x_path, None)
+        return x,y
+
+    def __data_generation(self,x_path,y_path):
+        if self.args.scale is not None and self.args.model_state.lower()!='train':
+            scl= self.args.scale
+            scl_h = int(self.dim_h*scl) if (self.dim_h*scl)%16==0 else \
+                int(((self.dim_h*scl) // 16 + 1) * 16)
+            scl_w = int(self.dim_w * scl) if (self.dim_w * scl) % 16 == 0 else \
+                int(((self.dim_h * scl) // 16 + 1) * 16)
+
+            x = np.empty((self.bs, scl_h, scl_w, 3), dtype="float32")
+        else:
+            x = np.empty((self.bs, self.dim_h, self.dim_w, 3), dtype="float32")
+        y = np.empty((self.bs, self.dim_h, self.dim_w, 1), dtype="float32")
+
+        for i,tmp_data in enumerate(x_path):
+            tmp_x_path = tmp_data
+            tmp_y_path = y_path[i] if not self.data_name.lower()=='classic' else None
+            tmp_x,tmp_y = self.transformer(tmp_x_path,tmp_y_path)
+            x[i,]=tmp_x
+            y[i,]=tmp_y
+
+        return x,y
+
+    def transformer(self, x_path, y_path):
+        tmp_x = cv.imread(x_path)
+        if y_path is not None:
+            tmp_y = cv.imread(y_path,cv.IMREAD_GRAYSCALE)
+        else:
+            tmp_y=None
+        h,w,_ = tmp_x.shape
+        if self.args.model_state == "train":
+            if self.args.crop_img:
+                i_h = random.randint(0,h-self.dim_h)
+                i_w = random.randint(0,w-self.dim_w)
+                tmp_x = tmp_x[i_h:i_h+self.dim_h,i_w:i_w+self.dim_w,]
+                tmp_y = tmp_y[i_h:i_h+self.dim_h,i_w:i_w+self.dim_w,]
+            else:
+                tmp_x = cv.resize(tmp_x,(self.dim_w,self.dim_h))
+                tmp_y = cv.resize(tmp_y,(self.dim_w,self.dim_h))
+        else:
+            if self.dim_w!=w and self.dim_h!=h:
+                tmp_x = cv.resize(tmp_x, (self.dim_w, self.dim_h))
+            if self.args.scale is not None:
+                scl = self.args.scale
+                scl_h = int(self.dim_h * scl) if (self.dim_h * scl) % 16 == 0 else \
+                    int(((self.dim_h * scl) // 16 + 1) * 16)
+                scl_w = int(self.dim_w * scl) if (self.dim_w * scl) % 16 == 0 else \
+                    int(((self.dim_h * scl) // 16 + 1) * 16)
+                tmp_x = cv.resize(tmp_x,dsize=(scl_w,scl_h))
+            if tmp_y is not None:
+                tmp_y = cv.resize(tmp_y, (self.dim_w, self.dim_h))
+
+        if tmp_y is not None:
+            tmp_y = np.expand_dims(np.float32(tmp_y)/255.,axis=-1)
+        tmp_x = np.float32(tmp_x)
+        return tmp_x, tmp_y
+
+    # def __read_h5(self,file_path):
+    #
+    #     with h5py.File(file_path,'r') as h5f:
+    #         # n_var = len(list(h5f.keys()))
+    #         data = np.array(h5f.get('data'))
+    #     return data
+
+def dataset_info(dataset_name, is_linux=False):
+
+    if is_linux:
+
+        config = {
+            'BSDS': {'img_height':400,# 321
+                     'img_width':400,#481
+                     'test_list': 'test_pair.lst',
+                     'data_dir': '/opt/dataset/BSDS',  # mean_rgb
+                        'yita': 0.5},
+            'BSDS300': {'img_height': 321,
+                     'img_width': 481,
+                    'test_list': 'test_pair.lst',
+                     'data_dir': '/opt/dataset/BSDS300',  # NIR
+                     'yita': 0.5},
+            'PASCAL': {'img_height':375,
+                        'img_width':500,
+                       'test_list': 'test_pair.lst',
+                       'data_dir': '/opt/dataset/PASCAL',  # mean_rgb
+                               'yita': 0.3},
+            'CID': {'img_height':512,
+                       'img_width':512,
+                    'test_list': 'test_pair.lst',
+                       'data_dir': '/opt/dataset/CID',  # mean_rgb
+                       'yita': 0.3},
+            'NYUD': {'img_height':425,
+                        'img_width':560,
+                     'test_list': 'test_pair.lst',
+                        'data_dir': '/opt/dataset/NYUD',  # mean_rgb
+                           'yita': 0.5},
+            'MULTICUE': {'img_height':720,
+                        'img_width':1280,
+                         'test_list': 'test_pair.lst',
+                         'data_dir': '/opt/dataset/MULTICUE',  # mean_rgb
+                              'yita': 0.3},
+            'BIPED': {'img_height': 720,
+                       'img_width': 1280,
+                      'test_list': 'test_rgb.lst',
+                      'train_list': 'train_rgb.lst',
+                       'data_dir': '/opt/dataset/BIPED/edges',  # WIN: '../.../dataset/BIPED/edges'
+                       'yita': 0.5},
+            'MBIPED': {'img_height': 720,
+                      'img_width': 1280,
+                      'test_list': 'test_rgbn.lst',
+                      'train_list': 'train_rgbn.lst',
+                      'data_dir': '/opt/dataset/BIPED/edges',  # WIN: '../.../dataset/BIPED/edges'
+                      'yita': 0.5},
+            'CLASSIC': {'img_height':512,  # 4032
+                      'img_width': 512, # 3024
+                      'test_list': None,
+                      'data_dir': 'data',  # mean_rgb
+                      'yita': 0.5},
+            'DCD': {'img_height': 336,# 240
+                      'img_width': 448,#360
+                    
+                      'test_list':'test_pair.lst',
+                      'data_dir': '/opt/dataset/DCD',  # mean_rgb
+                      'yita': 0.2}
+        }
+        data_info = config[dataset_name]
+        return data_info
+    else:
+        config = {
+            'BSDS': {'img_height': 512,#321
+                     'img_width': 512,#481
+                     'test_list': 'test_pair.lst',
+                     'data_dir': '../../dataset/BSDS',  # mean_rgb
+                     'yita': 0.5},
+            'BSDS300': {'img_height': 512,#321
+                        'img_width': 512,#481
+                        'test_list': 'test_pair.lst',
+                        'data_dir': '../../dataset/BSDS300',  # NIR
+                        'yita': 0.5},
+            'PASCAL': {'img_height': 375,
+                       'img_width': 500,
+                       'test_list': 'test_pair.lst',
+                       'data_dir': '/opt/dataset/PASCAL',  # mean_rgb
+                       'yita': 0.3},
+            'CID': {'img_height': 512,
+                    'img_width': 512,
+                    'test_list': 'test_pair.lst',
+                    'data_dir': '../../dataset/CID',  # mean_rgb
+                    'yita': 0.3},
+            'NYUD': {'img_height': 425,
+                     'img_width': 560,
+                     'test_list': 'test_pair.lst',
+                     'data_dir': '/opt/dataset/NYUD',  # mean_rgb
+                     'yita': 0.5},
+            'MULTICUE': {'img_height': 720,
+                         'img_width': 1280,
+                         'test_list': 'test_pair.lst',
+                         'data_dir': '../../dataset/MULTICUE',  # mean_rgb
+                         'yita': 0.3},
+            'BIPED': {'img_height': 720,#720
+                      'img_width': 1280,#1280
+                      'test_list': 'test_rgb.lst',
+                      'train_list': 'train_rgb.lst',
+                      'data_dir': '../../dataset/BIPED/edges',  # WIN: '../.../dataset/BIPED/edges'
+                      'yita': 0.5},
+            'CLASSIC': {'img_height': 512,
+                        'img_width': 512,
+                        'test_list': None,
+                        'train_list': None,
+                        'data_dir': 'data',  # mean_rgb
+                        'yita': 0.5},
+            'DCD': {'img_height': 240,
+                    'img_width': 360,
+                    'test_list': 'test_pair.lst',
+                    'data_dir': '/opt/dataset/DCD',  # mean_rgb
+                    'yita': 0.2}
+        }
+        data_info = config[dataset_name]
+        return data_info

DexiNed-TF2/mainTF.py

@@ -0,0 +1,99 @@
+#!/usr/bin/python3.7 python
+"""
+
+"""
+__author__ = "Xavier Soria Poma"
+__email__ = "[email protected] / [email protected]"
+__homepage__="www.cvc.uab.cat/people/xsoria"
+__credits__=["tensorflow_tutorial"]
+__copyright__   = "Copyright 2020, CIMI"
+
+import argparse
+import platform
+import tensorflow as tf
+
+from run_model import run_DexiNed
+from dataset_manager import dataset_info
+
+
+# Testing settings
+
+in_linux=True if platform.system()=="Linux" else False
+
+DATASET_NAME= ['BIPED','MBIPED','BSDS','BSDS300','CID','DCD','MULTICUE',
+                'PASCAL','NYUD','CLASSIC'] # 8
+TEST_DATA = DATASET_NAME[-1] # MULTICUE=6
+TRAIN_DATA = DATASET_NAME[0]
+test_data_inf = dataset_info(TEST_DATA, is_linux=in_linux)
+train_data_inf = dataset_info(TRAIN_DATA, is_linux=in_linux)
+test_model=False
+is_testing ="test" if test_model else "train"
+# training settings
+
+parser = argparse.ArgumentParser(description='Edge detection parameters for feeding the model')
+parser.add_argument("--train_dir",default=train_data_inf['data_dir'], help="path to folder containing images")
+parser.add_argument("--test_dir",default=test_data_inf['data_dir'], help="path to folder containing images")
+parser.add_argument("--data4train",default=TRAIN_DATA, type=str)
+parser.add_argument("--data4test",default=TEST_DATA, type=str)
+parser.add_argument('--train_list', default=train_data_inf['train_list'], type=str)  # SSMIHD: train_rgb_pair.lst, others train_pair.lst
+parser.add_argument('--test_list', default=test_data_inf['test_list'], type=str)  # SSMIHD: train_rgb_pair.lst, others train_pair.lst
+parser.add_argument("--model_state",default=is_testing , choices=["train", "test", "export"])
+parser.add_argument("--output_dir", default='results', help="where to put output files")
+parser.add_argument("--checkpoint_dir", default='checkpoints', help="directory with checkpoint to resume training from or use for testing")
+
+parser.add_argument('--model_name', default='DexiNed', choices=['DexiNed'])
+parser.add_argument('--continue_training', default=False, type=bool)
+parser.add_argument("--max_epochs", type=int,default=24, help="number of training epochs")#24
+parser.add_argument("--summary_freq", type=int, default=100, help="update summaries every summary_freq steps")
+parser.add_argument("--progress_freq", type=int, default=50, help="display progress every progress_freq steps")
+parser.add_argument("--display_freq", type=int, default=10, help="write current training images every display_freq steps")
+parser.add_argument("--scale", type=float, default=None, help="scale image before fed DexiNed.0.5, 1.5 ")
+
+parser.add_argument('--adjust_lr', default=[10, 15],type=int,
+                    help='Learning rate step size.')  # [5,10]BIRND [10,15]BIPED/BRIND
+parser.add_argument("--batch_size", type=int, default=8, help="number of images in batch")
+parser.add_argument("--test_bs", type=int, default=1, help="number of images in test batch")
+parser.add_argument("--batch_normalization", type=bool, default=True, help=" use batch norm")
+parser.add_argument("--image_height", type=int, default=352, help="scale images to this size before cropping to 256x256")
+parser.add_argument("--image_width", type=int, default=352, help="scale images to this size before cropping to 256x256")
+parser.add_argument("--crop_img", type=bool, default=False,
+                    help="4Training: True crop image, False resize image")
+parser.add_argument("--test_img_height", type=int, default=test_data_inf["img_height"],
+                    help="network input height size")
+parser.add_argument("--test_img_width", type=int, default=test_data_inf["img_width"],
+                    help="network input height size")
+
+parser.add_argument("--lr", type=float, default=0.0001, help=" learning rate for adam 1e-4")
+parser.add_argument("--beta1", type=float, default=0.5, help="momentum term of adam")
+parser.add_argument("--l1_weight", type=float, default=100.0, help="weight on L1 term for generator gradient")
+parser.add_argument("--gan_weight", type=float, default=1.0, help="weight on GAN term for generator gradient")
+parser.add_argument("--rgbn_mean", type=float, default=[103.939,116.779,123.68, 137.86], help="pixels mean")
+parser.add_argument("--checkpoint", type=str, default='DexiNed19_model.h5', help="checkpoint Name")
+
+arg = parser.parse_args()
+def main(args):
+    tf.debugging.set_log_device_placement(True)
+    gpus = tf.config.list_physical_devices('GPU')
+    if gpus:
+        # Restrict TensorFlow to only allocate 1GB of memory on the first GPU
+        try:
+            tf.config.set_logical_device_configuration(
+                gpus[0],
+                [tf.config.LogicalDeviceConfiguration(memory_limit=1024)])
+            logical_gpus = tf.config.list_logical_devices('GPU')
+            print(len(gpus), "Physical GPUs,", len(logical_gpus), "Logical GPUs")
+        except RuntimeError as e:
+            # Virtual devices must be set before GPUs have been initialized
+            print(e)
+
+    model = run_DexiNed(args=args)
+    if args.model_state=='train':
+        model.train()
+    elif args.model_state =='test':
+        model.test()
+    else:
+        raise NotImplementedError('Sorry you just can test or train the model, please set in '
+                                  'args.model_state=')
+
+if __name__=='__main__':
+    main(args=arg)

DexiNed-TF2/model.py

@@ -0,0 +1,346 @@
+from tensorflow.keras import callbacks
+from tensorflow.keras import layers, regularizers
+from tensorflow.keras import optimizers, metrics, losses
+from tensorflow.keras.models import Model
+from tensorflow.keras.models import load_model
+from tensorflow.keras import backend as K
+from tensorflow import keras
+import tensorflow as tf
+
+
+l2 = regularizers.l2
+w_decay=1e-8 #0.0#2e-4#1e-3, 2e-4 # please define weight decay
+K.clear_session()
+# weight_init = tf.initializers.RandomNormal(mean=0.,stddev=0.01)
+# weight_init = tf.initializers.glorot_normal()
+weight_init = tf.initializers.glorot_normal()
+
+class _DenseLayer(layers.Layer):
+    """_DenseBlock model.
+
+       Arguments:
+         out_features: number of output features
+    """
+
+    def __init__(self, out_features,**kwargs):
+        super(_DenseLayer, self).__init__(**kwargs)
+        k_reg = None if w_decay is None else l2(w_decay)
+        self.layers = []
+        self.layers.append(tf.keras.Sequential(
+            [
+                layers.ReLU(),
+                layers.Conv2D(
+                    filters=out_features, kernel_size=(3,3), strides=(1,1), padding='same',
+                    use_bias=True, kernel_initializer=weight_init,
+                kernel_regularizer=k_reg),
+                layers.BatchNormalization(),
+                layers.ReLU(),
+                layers.Conv2D(
+                    filters=out_features, kernel_size=(3,3), strides=(1,1), padding='same',
+                    use_bias=True, kernel_initializer=weight_init,
+                    kernel_regularizer=k_reg),
+                layers.BatchNormalization(),
+            ])) # first relu can be not needed
+
+
+    def call(self, inputs):
+        x1, x2 = tuple(inputs)
+        new_features = x1
+        for layer in self.layers:
+            new_features = layer(new_features)
+
+        return 0.5 * (new_features + x2), x2
+
+
+class _DenseBlock(layers.Layer):
+    """DenseBlock layer.
+
+       Arguments:
+         num_layers: number of _DenseLayer's per block
+         out_features: number of output features
+    """
+
+    def __init__(self,
+                 num_layers,
+                 out_features,**kwargs):
+        super(_DenseBlock, self).__init__(**kwargs)
+        self.layers = [_DenseLayer(out_features) for i in range(num_layers)]
+
+    def call(self, inputs):
+        for layer in self.layers:
+            inputs = layer(inputs)
+
+        return inputs
+
+
+class UpConvBlock(layers.Layer):
+    """UpConvDeconvBlock layer.
+
+       Arguments:
+         up_scale: int
+    """
+
+    def __init__(self, up_scale,**kwargs):
+        super(UpConvBlock, self).__init__(**kwargs)
+        constant_features = 16
+        k_reg = None if w_decay is None else l2(w_decay)
+        features = []
+        total_up_scale = 2 ** up_scale
+        for i in range(up_scale):
+            out_features = 1 if i == up_scale-1 else constant_features
+            if i==up_scale-1:
+                features.append(layers.Conv2D(
+                    filters=out_features, kernel_size=(1,1), strides=(1,1), padding='same',
+                    activation='relu', kernel_initializer=tf.initializers.RandomNormal(mean=0.),
+                    kernel_regularizer=k_reg,use_bias=True)) #tf.initializers.TruncatedNormal(mean=0.)
+                features.append(layers.Conv2DTranspose(
+                    out_features, kernel_size=(total_up_scale,total_up_scale),
+                    strides=(2,2), padding='same',
+                    kernel_initializer=tf.initializers.RandomNormal(stddev=0.1),
+                    kernel_regularizer=k_reg,use_bias=True)) # stddev=0.1
+            else:
+
+                features.append(layers.Conv2D(
+                    filters=out_features, kernel_size=(1,1), strides=(1,1), padding='same',
+                    activation='relu',kernel_initializer=weight_init,
+                kernel_regularizer=k_reg,use_bias=True))
+                features.append(layers.Conv2DTranspose(
+                    out_features, kernel_size=(total_up_scale,total_up_scale),
+                    strides=(2,2), padding='same', use_bias=True,
+                    kernel_initializer=weight_init, kernel_regularizer=k_reg))
+
+        self.features = keras.Sequential(features)
+
+    def call(self, inputs):
+        return self.features(inputs)
+
+
+class SingleConvBlock(layers.Layer):
+    """SingleConvBlock layer.
+
+       Arguments:
+         out_features: number of output features
+         stride: stride per convolution
+    """
+
+    def __init__(self, out_features, k_size=(1,1),stride=(1,1),
+                 use_bs=False, use_act=False,w_init=None,**kwargs): # bias_init=tf.constant_initializer(0.0)
+        super(SingleConvBlock, self).__init__(**kwargs)
+        self.use_bn = use_bs
+        self.use_act = use_act
+        k_reg = None if w_decay is None else l2(w_decay)
+        self.conv = layers.Conv2D(
+            filters=out_features, kernel_size=k_size, strides=stride,
+            padding='same',kernel_initializer=w_init,
+            kernel_regularizer=k_reg)#, use_bias=True, bias_initializer=bias_init
+        if self.use_bn:
+            self.bn = layers.BatchNormalization()
+        if self.use_act:
+            self.relu = layers.ReLU()
+
+    def call(self, inputs):
+        x =self.conv(inputs)
+        if self.use_bn:
+            x = self.bn(x)
+        if self.use_act:
+            x = self.relu(x)
+        return x
+
+
+class DoubleConvBlock(layers.Layer):
+    """DoubleConvBlock layer.
+
+       Arguments:
+         mid_features: number of middle features
+         out_features: number of output features
+         stride: stride per mid-layer convolution
+    """
+
+    def __init__(self, mid_features, out_features=None, stride=(1,1),
+                 use_bn=True,use_act=True,**kwargs):
+        super(DoubleConvBlock, self).__init__(**kwargs)
+        self.use_bn =use_bn
+        self.use_act =use_act
+        out_features = mid_features if out_features is None else out_features
+        k_reg = None if w_decay is None else l2(w_decay)
+
+        self.conv1 = layers.Conv2D(
+            filters=mid_features, kernel_size=(3, 3), strides=stride, padding='same',
+        use_bias=True, kernel_initializer=weight_init,
+        kernel_regularizer=k_reg)
+        self.bn1 = layers.BatchNormalization()
+
+        self.conv2 = layers.Conv2D(
+            filters=out_features, kernel_size=(3, 3), padding='same',strides=(1,1),
+        use_bias=True, kernel_initializer=weight_init,
+        kernel_regularizer=k_reg)
+        self.bn2 = layers.BatchNormalization()
+        self.relu = layers.ReLU()
+
+    def call(self, inputs):
+        x = self.conv1(inputs)
+        x = self.bn1(x)
+        x = self.relu(x)
+        x = self.conv2(x)
+        x = self.bn2(x)
+        if self.use_act:
+            x = self.relu(x)
+        return x
+
+
+class DexiNed(tf.keras.Model):
+    """DexiNet model."""
+
+    def __init__(self,rgb_mean=None,
+                 **kwargs):
+        super(DexiNed, self).__init__(**kwargs)
+
+        self.rgbn_mean = rgb_mean
+        self.block_1 = DoubleConvBlock(32, 64, stride=(2,2),use_act=False)
+        self.block_2 = DoubleConvBlock(128,use_act=False)
+        self.dblock_3 = _DenseBlock(2, 256)
+        self.dblock_4 = _DenseBlock(3, 512)
+        self.dblock_5 = _DenseBlock(3, 512)
+        self.dblock_6 = _DenseBlock(3, 256)
+        self.maxpool = layers.MaxPool2D(pool_size=(3, 3), strides=2, padding='same')
+
+        # left skip connections, figure in Journal
+        self.side_1 = SingleConvBlock(128,k_size=(1,1),stride=(2,2),use_bs=True,
+                                      w_init=weight_init)
+        self.side_2 = SingleConvBlock(256,k_size=(1,1),stride=(2,2),use_bs=True,
+                                      w_init=weight_init)
+        self.side_3 = SingleConvBlock(512,k_size=(1,1),stride=(2,2),use_bs=True,
+                                      w_init=weight_init)
+        self.side_4 = SingleConvBlock(512,k_size=(1,1),stride=(1,1),use_bs=True,
+                                      w_init=weight_init)
+        # self.side_5 = SingleConvBlock(256,k_size=(1,1),stride=(1,1),use_bs=True,
+        #                               w_init=weight_init)
+
+        # right skip connections, figure in Journal paper
+        self.pre_dense_2 = SingleConvBlock(256,k_size=(1,1),stride=(2,2),
+                                      w_init=weight_init) # use_bn=True
+        self.pre_dense_3 = SingleConvBlock(256,k_size=(1,1),stride=(1,1),use_bs=True,
+                                      w_init=weight_init)
+        self.pre_dense_4 = SingleConvBlock(512,k_size=(1,1),stride=(1,1),use_bs=True,
+                                      w_init=weight_init)
+        # self.pre_dense_5_0 = SingleConvBlock(512, k_size=(1,1),stride=(2,2),
+        #                               w_init=weight_init) # use_bn=True
+        self.pre_dense_5 = SingleConvBlock(512,k_size=(1,1),stride=(1,1),use_bs=True,
+                                      w_init=weight_init)
+        self.pre_dense_6 = SingleConvBlock(256,k_size=(1,1),stride=(1,1),use_bs=True,
+                                      w_init=weight_init)
+        # USNet
+        self.up_block_1 = UpConvBlock(1)
+        self.up_block_2 = UpConvBlock(1)
+        self.up_block_3 = UpConvBlock(2)
+        self.up_block_4 = UpConvBlock(3)
+        self.up_block_5 = UpConvBlock(4)
+        self.up_block_6 = UpConvBlock(4)
+
+        self.block_cat = SingleConvBlock(
+            1,k_size=(1,1),stride=(1,1),
+            w_init=tf.constant_initializer(1/5))
+
+
+    def slice(self, tensor, slice_shape):
+        height, width = slice_shape
+        return tensor[..., :height, :width]
+
+
+    def call(self, x):
+        # Block 1
+        x = x-self.rgbn_mean[:-1]
+        block_1 = self.block_1(x)
+        block_1_side = self.side_1(block_1)
+
+        # Block 2
+        block_2 = self.block_2(block_1)
+        block_2_down = self.maxpool(block_2) # the key for the second skip connec...
+        block_2_add = block_2_down + block_1_side
+        block_2_side = self.side_2(block_2_add) #
+
+        # Block 3
+        block_3_pre_dense = self.pre_dense_3(block_2_down)
+        block_3, _ = self.dblock_3([block_2_add, block_3_pre_dense])
+        block_3_down = self.maxpool(block_3)
+        block_3_add = block_3_down + block_2_side
+        block_3_side = self.side_3(block_3_add)
+
+        # Block 4
+        block_4_pre_dense_256 = self.pre_dense_2(block_2_down)
+        block_4_pre_dense = self.pre_dense_4(block_4_pre_dense_256 + block_3_down)
+        block_4, _ = self.dblock_4([block_3_add, block_4_pre_dense])
+        block_4_down = self.maxpool(block_4)
+        block_4_add = block_4_down + block_3_side
+        block_4_side = self.side_4(block_4_add)
+
+        # Block 5
+        # block_5_pre_dense_512 = self.pre_dense_5_0(block_4_pre_dense_256)
+        block_5_pre_dense = self.pre_dense_5(block_4_down )
+        block_5, _ = self.dblock_5([block_4_add, block_5_pre_dense])
+        block_5_add = block_5 + block_4_side
+
+        # Block 6
+        block_6_pre_dense = self.pre_dense_6(block_5)
+        block_6, _ = self.dblock_6([block_5_add, block_6_pre_dense])
+
+        # upsampling blocks
+        height, width = x.shape[1:3]
+        slice_shape = (height, width)
+        out_1 = self.up_block_1(block_1) # self.slice(, slice_shape)
+        out_2 = self.up_block_2(block_2)
+        out_3 = self.up_block_3(block_3)
+        out_4 = self.up_block_4(block_4)
+        out_5 = self.up_block_5(block_5)
+        out_6 = self.up_block_6(block_6)
+        results = [out_1, out_2, out_3, out_4, out_5, out_6]
+
+        # concatenate multiscale outputs
+        block_cat = tf.concat(results, 3)  # BxHxWX6
+        block_cat = self.block_cat(block_cat)  # BxHxWX1
+
+        results.append(block_cat)
+
+        return results
+
+def weighted_cross_entropy_loss(input, label):
+    y = tf.cast(label,dtype=tf.float32)
+    negatives = tf.math.reduce_sum(1.-y)
+    positives = tf.math.reduce_sum(y)
+
+    beta = negatives/(negatives + positives)
+    pos_w = beta/(1-beta)
+    cost = tf.nn.weighted_cross_entropy_with_logits(
+    labels=label, logits=input, pos_weight=pos_w, name=None)
+    cost = tf.reduce_sum(cost*(1-beta))
+    return tf.where(tf.equal(positives, 0.0), 0.0, cost)
+
+
+def pre_process_binary_cross_entropy(bc_loss,input, label,arg, use_tf_loss=False):
+    # preprocess data
+    y = label
+    loss = 0
+    w_loss=1.0
+    preds = []
+    for tmp_p in input:
+        # tmp_p = input[i]
+
+        # loss processing
+        tmp_y = tf.cast(y, dtype=tf.float32)
+        mask = tf.dtypes.cast(tmp_y > 0., tf.float32)
+        b,h,w,c=mask.get_shape()
+        positives = tf.math.reduce_sum(mask, axis=[1, 2, 3], keepdims=True)
+        negatives = h*w*c-positives
+
+        beta2 = (1.*positives) / (negatives + positives) # negatives in hed
+        beta = (1.1*negatives)/ (positives + negatives) # positives in hed
+        pos_w = tf.where(tf.equal(y, 0.0), beta2, beta)
+        logits = tf.sigmoid(tmp_p)
+
+        l_cost = bc_loss(y_true=tmp_y, y_pred=logits,
+                         sample_weight=pos_w)
+
+        preds.append(logits)
+        loss += (l_cost*w_loss)
+
+    return preds, loss

DexiNed-TF2/run_model.py

@@ -0,0 +1,234 @@
+from __future__ import absolute_import, division, print_function
+
+import time, os
+import numpy as np
+from os.path import join
+import cv2 as cv
+
+from model import *
+from utls import image_normalization,visualize_result, tensor2image, cv_imshow,h5_writer
+from dataset_manager import DataLoader
+
+
+BUFFER_SIZE = 448
+
+# tf.set_random_seed(1)
+class run_DexiNed():
+
+    def __init__(self, args):
+        self.model_state= args.model_state
+        self.args = args
+        self.img_width=args.image_width
+        self.img_height = args.image_height
+        self.epochs = args.max_epochs
+        self.bs = args.batch_size
+
+    def train(self):
+        # Validation and Train dataset generation
+
+        train_data = DataLoader(data_name=self.args.data4train, arg=self.args)
+        n_train =train_data.indices.size #data_cache["n_files"]
+        val_data = DataLoader(data_name=self.args.data4train,
+                              arg=self.args, is_val=True)
+        val_idcs = np.arange(val_data.indices.size)
+        # Summary and checkpoint manager
+        model_dir =self.args.model_name+'2'+self.args.data4train
+        summary_dir = os.path.join('logs',model_dir)
+        train_log_dir=os.path.join(summary_dir,'train')
+        val_log_dir =os.path.join(summary_dir,'test')
+
+        checkpoint_dir = os.path.join(self.args.checkpoint_dir,model_dir)
+        epoch_ckpt_dir = checkpoint_dir + 'epochs'
+        os.makedirs(epoch_ckpt_dir, exist_ok=True)
+        os.makedirs(train_log_dir,exist_ok=True)
+        os.makedirs(val_log_dir,exist_ok=True)
+        os.makedirs(checkpoint_dir, exist_ok=True)
+
+        train_writer = tf.summary.create_file_writer(train_log_dir)
+        val_writer = tf.summary.create_file_writer(val_log_dir)
+
+        my_model = DexiNed(rgb_mean=self.args.rgbn_mean)#rgb_mean=self.args.rgbn_mean
+
+        # accuracy = metrics.SparseCategoricalAccuracy()
+        accuracy = metrics.BinaryAccuracy()
+        accuracy_val = metrics.BinaryAccuracy()
+        loss_bc = losses.BinaryCrossentropy()
+        optimizer = optimizers.Adam(
+            learning_rate=self.args.lr, beta_1=self.args.beta1)
+        iter = 0
+
+        imgs_res_folder = os.path.join(self.args.output_dir, "current_training")
+        os.makedirs(imgs_res_folder, exist_ok=True)
+        global_loss = 1000.
+        t_loss = []
+        ckpt_save_mode = "h5"
+        tmp_lr = self.args.lr
+        for epoch in range(self.args.max_epochs):
+            # training
+            t_loss = []
+            # if epoch in self.args.adjust_lr:
+            tmp_lr=tmp_lr*0.1
+            optimizer.lr.assign(tmp_lr)
+            for step, (x, y) in enumerate(train_data):
+
+                with tf.GradientTape() as tape:
+                    pred = my_model(x, training=True)
+
+                    preds, loss = pre_process_binary_cross_entropy(
+                        loss_bc, pred, y, self.args, use_tf_loss=False)
+
+                accuracy.update_state(y_true=y, y_pred=preds[-1])
+                gradients = tape.gradient(loss, my_model.trainable_variables)
+                optimizer.apply_gradients(zip(gradients, my_model.trainable_variables))
+
+                # logging the current accuracy value so far.
+                t_loss.append(loss.numpy())
+                if step % 10 == 0:
+                    print("Epoch:", epoch, "Step:", step, "Loss: %.4f" % loss.numpy(),
+                          "Accuracy: %.4f" % accuracy.result(), time.ctime())
+
+                if step % 10 == 0:
+                    # visualize preds
+                    img_test = 'Epoch: {0} Sample {1}/{2} Loss: {3}' \
+                        .format(epoch, step, n_train // self.args.batch_size, loss.numpy())
+                    vis_imgs = visualize_result(
+                        x=x[2], y=y[2], p=preds, img_title=img_test)
+                    cv.imwrite(os.path.join(imgs_res_folder, 'results.png'), vis_imgs)
+                if step % 20 == 0 and loss < global_loss:  # 500
+                    if epoch==0 and step==0:
+                        tmp_loss = np.array(t_loss)
+                        with train_writer.as_default():
+                            tf.summary.scalar('loss', tmp_loss.mean(), step=epoch)
+                            tf.summary.scalar('accuracy', accuracy.result(), step=epoch)
+
+                    save_ckpt_path = os.path.join(checkpoint_dir, "DexiNedL_model.h5")
+                    Model.save_weights(my_model, save_ckpt_path, save_format='h5')
+
+                    global_loss = loss
+                    print("Model saved in:  ", save_ckpt_path, "Current loss:", global_loss.numpy())
+
+                iter += 1  # global iteration
+
+            t_loss = np.array(t_loss)
+            # train summary
+            if epoch!=0:
+                with train_writer.as_default():
+                    tf.summary.scalar('loss', t_loss.mean(), step=epoch)
+                    tf.summary.scalar('accuracy', accuracy.result(), step=epoch)
+
+            Model.save_weights(my_model, os.path.join(epoch_ckpt_dir, "DexiNed{}_model.h5".format(str(epoch))),
+                               save_format=ckpt_save_mode)
+            print("Epoch:", epoch, "Model saved in Loss: ", t_loss.mean())
+
+            # validation
+            t_val_loss = []
+            for i, (x_val, y_val) in enumerate(val_data):
+
+                pred_val = my_model(x_val)
+                v_logits, V_loss = pre_process_binary_cross_entropy(
+                    loss_bc, pred_val, y_val, self.args, use_tf_loss=False)
+                accuracy_val.update_state(y_true=y_val, y_pred=v_logits[-1])
+                t_val_loss.append(V_loss.numpy())
+                if i == 7:
+                    break
+            val_acc = accuracy_val.result()
+            t_val_loss = np.array(t_val_loss)
+            print("Epoch(validation):", epoch, "Val loss: ", t_val_loss.mean(),
+                  "Accuracy: ", val_acc.numpy())
+            # validation summary
+            with val_writer.as_default():
+                tf.summary.scalar('loss', t_val_loss.mean(), step=epoch)
+                tf.summary.scalar('accuracy', val_acc.numpy(), step=epoch)
+
+            # Reset metrics every epoch
+            accuracy.reset_states()
+            accuracy_val.reset_states()
+
+        my_model.summary()
+
+
+    def test(self):
+        # Test dataset generation
+
+        test_data = DataLoader(data_name=self.args.data4test, arg=self.args)
+        n_test = test_data.indices.size  # data_cache["n_files"]
+
+        optimizer = tf.keras.optimizers.Adam(
+            learning_rate=self.args.lr, beta_1=self.args.beta1)
+
+        my_model = DexiNed(rgb_mean=self.args.rgbn_mean)
+        input_shape = test_data.input_shape
+        my_model.build(input_shape=input_shape)  # rgb_mean=self.args.rgbn_mean
+
+        checkpoit_dir = os.path.join(self.args.checkpoint_dir,
+                                     self.args.model_name + "2" + self.args.data4train)
+
+        my_model.load_weights(os.path.join(checkpoit_dir, self.args.checkpoint))
+
+        result_dir = os.path.join(
+            self.args.output_dir,
+            self.args.model_name + '-' + self.args.data4train + "2" + self.args.data4test)
+        os.makedirs(result_dir, exist_ok=True)
+        if self.args.scale is not None:
+            scl = self.args.scale
+            save_dir = ['fuse_'+str(scl), 'avrg_'+str(scl), 'h5_'+str(scl)]
+        else:
+            save_dir = ['fuse', 'avrg', 'h5']
+        save_dirs = []
+        for tmp_dir in save_dir:
+            os.makedirs(os.path.join(result_dir, tmp_dir), exist_ok=True)
+            save_dirs.append(os.path.join(result_dir, tmp_dir))
+
+        total_time = []
+        data_names = test_data.imgs_name
+        data_shape = test_data.imgs_shape
+        k = 0
+        for step, (x, y) in enumerate(test_data):
+
+            start_time = time.time()
+            preds = my_model(x, training=False)
+            tmp_time = time.time() - start_time
+            total_time.append(tmp_time)
+
+            preds = [tf.sigmoid(i).numpy() for i in preds]
+            all_preds = np.array(preds)
+            for i in range(all_preds.shape[1]):
+                tmp_name = data_names[k]
+                tmp_name, _ = os.path.splitext(tmp_name)
+                tmp_shape = data_shape[k]
+
+                tmp_preds = all_preds[:, i, ...]
+                tmp_av = np.expand_dims(tmp_preds.mean(axis=0), axis=0)
+                tmp_preds = np.concatenate((tmp_preds, tmp_av), axis=0)
+                res_preds = []
+                for j in range(tmp_preds.shape[0]):
+                    tmp_pred = tmp_preds[j, ...]
+                    tmp_pred[tmp_pred < 0.0] = 0.0
+                    tmp_pred = cv.bitwise_not(np.uint8(image_normalization(tmp_pred)))
+                    h, w = tmp_pred.shape[:2]
+                    if h != tmp_shape[0] or w != tmp_shape[1]:
+                        tmp_pred = cv.resize(tmp_pred, (tmp_shape[1], tmp_shape[0]))
+                    res_preds.append(tmp_pred)
+                n_save =len(tmp_preds)-2
+                for idx in range(len(save_dirs) - 1):
+                    s_dir = save_dirs[idx]
+                    tmp = res_preds[n_save + idx]
+                    cv.imwrite(join(s_dir, tmp_name + '.png'), tmp)
+                h5_writer(path=join(save_dirs[-1], tmp_name + '.h5'),
+                          vars=np.squeeze(res_preds))
+                print("saved:", join(save_dirs[-1], tmp_name + '.h5'), tmp_preds.shape)
+                k += 1
+
+            # tmp_name = data_names[step][:-3]+"png"
+            # tmp_shape = data_shape[step]
+            # tmp_path = os.path.join(result_dir,tmp_name)
+            # tensor2image(preds[-1].numpy(), img_path =tmp_path,img_shape=tmp_shape)
+
+        total_time = np.array(total_time)
+
+        print('-------------------------------------------------')
+        print("End testing in: ", self.args.data4test)
+        print("Batch size: ", self.args.test_bs)
+        print("Time average per image: ", total_time.mean(), "secs")
+        print("Total time: ", total_time.sum(), "secs")
+        print('-------------------------------------------------')

DexiNed-TF2/utls.py

@@ -0,0 +1,257 @@
+
+import numpy as np
+import os
+import cv2 as cv
+import h5py
+
+def image_normalization(img, img_min=0, img_max=255):
+    """This is a typical image normalization function
+    where the minimum and maximum of the image is needed
+    source: https://en.wikipedia.org/wiki/Normalization_(image_processing)
+    :param img: an image could be gray scale or color
+    :param img_min:  for default is 0
+    :param img_max: for default is 255
+    :return: a normalized image, if max is 255 the dtype is uint8
+    """
+    img = np.float32(img)
+    epsilon=1e-12 # whenever an inconsistent image
+    img = (img-np.min(img))*(img_max-img_min)/((np.max(img)-np.min(img))+epsilon)+img_min
+    return img
+
+def make_dirs(paths): # make path or paths dirs
+    if not os.path.exists(paths):
+        os.makedirs(paths)
+        print("Directories have been created: ",paths)
+        return True
+    else:
+        print("Directories already exists: ", paths)
+        return False
+
+def read_files_list(list_path,dataset_name=None):
+    mfiles = open(list_path)
+    file_names = mfiles.readlines()
+    mfiles.close()
+
+    file_names = [f.strip() for f in file_names] # this is for delete '\n'
+    return file_names
+
+def data_parser(dataset_dir,dataset_name, list_name=None,training=True):
+
+    if dataset_name.upper()!="CLASSIC":
+        files_name = list_name # dataset base dir
+        list_path = os.path.join(dataset_dir, files_name)
+        data_list = read_files_list(list_path)
+        tmp_list = data_list[0]
+        tmp_list = tmp_list.split(' ')
+        n_lists = len(tmp_list)
+    if dataset_name.upper()=='BIPED':
+        in_dir = os.path.join(dataset_dir,'imgs','train') if training else \
+            os.path.join(dataset_dir, 'imgs', 'test')
+        gt_dir = os.path.join(dataset_dir,'edge_maps','train') if training else \
+            os.path.join(dataset_dir, 'edge_maps', 'test')
+    elif dataset_name.upper()=="CLASSIC":
+        list_path= dataset_dir
+        data_list = os.listdir(dataset_dir)
+        data_list = [os.path.join(dataset_dir,i)for i in data_list]
+        n_lists = None
+    else:
+
+        in_dir = dataset_dir
+        gt_dir = dataset_dir
+
+    if n_lists==1:
+        data_list = [c.split(' ') for c in data_list]
+        data_list = [(os.path.join(in_dir, c[0])) for c in data_list]
+    elif n_lists==2:
+        data_list = [c.split(' ') for c in data_list]
+        data_list = [(os.path.join(in_dir, c[0]),
+                       os.path.join(gt_dir, c[1])) for c in data_list]
+    else:
+        print('There are just two entry files, dataset:', dataset_name)
+
+    num_data = len(data_list)
+    print(" Enterely training set-up from {}, size: {}".format(list_path, num_data))
+
+    all_train_ids = np.arange(num_data)
+    np.random.shuffle(all_train_ids)
+    if training:
+
+        train_ids = all_train_ids[:int(0.9 * len(data_list))]
+        valid_ids = all_train_ids[int(0.9 * len(data_list)):]
+
+        print("Training set-up from {}, size: {}".format(list_path, len(train_ids)))
+        print("Validation set-up from {}, size: {}".format(list_path, len(valid_ids)))
+        train_list = [data_list[i] for i in train_ids]
+        val_list = [data_list[i] for i in valid_ids]
+        cache_info = {
+            "files_path": data_list,
+            "train_paths":train_list,
+            "val_paths": val_list,
+            "n_files": num_data,
+            "train_indices": train_ids,
+            "val_indices": valid_ids
+        }
+    else:
+        data_shape = []
+        data_name=[]
+        for tmp_path in data_list:
+            tmp_img = cv.imread(tmp_path[0]) if len(tmp_path)<4 else cv.imread(tmp_path)
+            tmp_name = os.path.basename(tmp_path[0])if len(tmp_path)<4 else\
+                os.path.basename(tmp_path)
+            tmp_shape = tmp_img.shape[:2]
+            data_shape.append(tmp_shape)
+            data_name.append(tmp_name)
+            is_gt = True if len(tmp_path)<4 else False
+
+        print("Testing set-up from {}, size: {}".format(list_path, len(all_train_ids)))
+        cache_info = {
+            "files_path": data_list,
+            "n_files": num_data,
+            "is_gt":is_gt,
+            "data_name": data_name,
+            "data_shape":data_shape
+        }
+
+    return cache_info
+
+def cv_imshow(title='image',img=None):
+
+    cv.imshow(title,img)
+    cv.waitKey(0)
+    cv.destroyAllWindows()
+
+def image_normalization(img, img_min=0, img_max=255):
+    """This is a typical image normalization function
+    where the minimum and maximum of the image is needed
+    source: https://en.wikipedia.org/wiki/Normalization_(image_processing)
+    :param img: an image could be gray scale or color
+    :param img_min:  for default is 0
+    :param img_max: for default is 255
+    :return: a normalized image, if max is 255 the dtype is uint8
+    """
+    img = np.float32(img)
+    epsilon=1e-12 # whenever an inconsistent image
+    img = (img-np.min(img))*(img_max-img_min)/((np.max(img)-np.min(img))+epsilon)+img_min
+    return img
+
+def restore_rgb(config,I, restore_rgb=False):
+    """
+    :param config: [args.channel_swap, args.mean_pixel_value]
+    :param I: and image or a set of images
+    :return: an image or a set of images restored
+    """
+
+    if  len(I)>3 and not type(I)==np.ndarray:
+        I =np.array(I)
+        I = I[:,:,:,0:3]
+        n = I.shape[0]
+        for i in range(n):
+            x = I[i,...]
+            x = np.array(x, dtype=np.float32)
+            x += config[1]
+            if restore_rgb:
+                x = x[:, :, config[0]]
+            x = image_normalization(x)
+            I[i,:,:,:]=x
+    elif len(I.shape)==3 and I.shape[-1]==3:
+        I = np.array(I, dtype=np.float32)
+        I += config[1]
+        if restore_rgb:
+            I = I[:, :, config[0]]
+        I = image_normalization(I)
+    else:
+        print("Sorry the input data size is out of our configuration")
+    # print("The enterely I data {} restored".format(I.shape))
+    return I
+
+def visualize_result(x,y,p, img_title):
+    """
+    function for tensorflow results
+    :param imgs_list: a list of prediction, gt and input data
+    :param arg:
+    :return: one image with the whole of imgs_list data
+    """
+    imgs_list = []
+    imgs_list.append(x)
+    imgs_list.append(y)
+    for i in p:
+        tmp = i.numpy()[2]
+        imgs_list.append(tmp)
+
+    n_imgs = len(imgs_list)
+    data_list =[]
+    for i in range(n_imgs):
+        tmp = imgs_list[i]
+        if tmp.shape[-1]==3:
+            # tmp = np.transpose(np.squeeze(tmp),[1,2,0])
+            # tmp=restore_rgb([arg.channel_swap,arg.mean_pixel_values[:3]],tmp)
+            tmp = tmp[:,:,[2,1,0]]
+            tmp = np.uint8(image_normalization(tmp))
+        else:
+            tmp= np.squeeze(tmp)
+            if len(tmp.shape) == 2:
+                tmp = np.uint8(image_normalization(tmp))
+                tmp = cv.bitwise_not(tmp)
+                tmp = cv.cvtColor(tmp, cv.COLOR_GRAY2BGR)
+            else:
+                tmp = np.uint8(image_normalization(tmp))
+
+
+        data_list.append(tmp)
+    img = data_list[0]
+    if n_imgs % 2 == 0:
+        imgs = np.zeros((img.shape[0] * 2 + 10, img.shape[1] * (n_imgs // 2) + ((n_imgs // 2 - 1) * 5), 3))
+    else:
+        imgs = np.zeros((img.shape[0] * 2 + 10, img.shape[1] * ((1 + n_imgs) // 2) + ((n_imgs // 2) * 5), 3))
+        n_imgs += 1
+
+    k=0
+    imgs = np.uint8(imgs)
+    i_step = img.shape[0]+10
+    j_step = img.shape[1]+5
+    for i in range(2):
+        for j in range(n_imgs//2):
+            if k<len(data_list):
+                imgs[i*i_step:i*i_step+img.shape[0],j*j_step:j*j_step+img.shape[1],:]=data_list[k]
+                k+=1
+            else:
+                pass
+
+    # last processing, resize and add title
+    vis_imgs = cv.resize(imgs, (int(imgs.shape[1] * 0.8), int(imgs.shape[0] * 0.8)))
+    BLACK = (0, 0, 255)
+    font = cv.FONT_HERSHEY_SIMPLEX
+    font_size = 1.1
+    font_color = BLACK
+    font_thickness = 2
+    x, y = 30, 30
+    vis_imgs = cv.putText(vis_imgs, img_title, (x, y), font, font_size, font_color, font_thickness,
+                          cv.LINE_AA)
+    return vis_imgs
+
+def tensor2image(image, img_path =None,img_shape=None):
+    tmp_img = np.squeeze(image_normalization(image))
+    tmp_img = np.uint8(cv.resize(tmp_img,(img_shape[1],img_shape[0])))
+    tmp_img = cv.bitwise_not(tmp_img)
+    cv.imwrite(img_path,tmp_img)
+    print("Prediction saved in:",img_path)
+
+def tensor2image(image, img_path =None,img_shape=None):
+    tmp_img = np.squeeze(image_normalization(image))
+    tmp_img = np.uint8(cv.resize(tmp_img,(img_shape[1],img_shape[0])))
+    tmp_img = cv.bitwise_not(tmp_img)
+    cv.imwrite(img_path,tmp_img)
+    print("Prediction saved in:",img_path)
+
+def h5_writer(vars,path, is_multivar=False):
+    # vars is a single variable o a list of multiple variables
+
+    with h5py.File(path, 'w') as hf:
+        if is_multivar:
+            assert isinstance(vars, list)
+            for idx, var in enumerate(vars):
+                hf.create_dataset('data'+str(idx+1), data=var)
+            # print('Set of data',len(vars),'saved in ', path)
+        else:
+            hf.create_dataset('data', data=vars)
+            # print("Data [", len(vars), "] saved in: ", path)

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2019 Xavier Soria Poma
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -1,19 +1,204 @@
----
-title: DexinedApp
-emoji: 🚀
-colorFrom: red
-colorTo: red
-sdk: docker
-app_port: 8501
-tags:
-- streamlit
-pinned: false
-short_description: App that implemented dexined
----
-
-# Welcome to Streamlit!
-
-Edit `/src/streamlit_app.py` to customize this app to your heart's desire. :heart:
-
-If you have any questions, checkout our [documentation](https://docs.streamlit.io) and [community
-forums](https://discuss.streamlit.io).
+[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/dense-extreme-inception-network-for-edge/edge-detection-on-biped-1)](https://paperswithcode.com/sota/edge-detection-on-biped-1?p=dense-extreme-inception-network-for-edge)
+[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/dense-extreme-inception-network-for-edge/edge-detection-on-mdbd)](https://paperswithcode.com/sota/edge-detection-on-mdbd?p=dense-extreme-inception-network-for-edge)
+# DexiNed: Dense Extreme Inception Network for Edge Detection (Extended version)
+
+```diff
+- We have just updated the last version of DexiNed in Pytorch.
+- The implementation with TF1 is fransfered to the "legacy" dir.
+- These days we are going to updated minor changes but the model, 
+  loss function, and the testing and training procedures does not need changes,
+  hopefully :), if you have time please check errors and let me know
+  in Issues.
+
+```
+
+This is the extended work from DexiNed presented in WACV2020. 
+We have improved the architecture and the dataset. Unlike of the state-of-the-art CNN
+based edge detectors, this models has a single training stage, 
+but it is still able to overcome those models in edge detection datasets. Moreover, 
+Dexined does not need pre-trained weights, and it is trained from the scratch 
+with fewer parameters tunning. To know more about DexiNed, read our first 
+version of Dexined in [ieeexplore](https://ieeexplore.ieee.org/document/9093290), 
+and its extended version in [arXiv](https://arxiv.org/abs/2112.02250).
+The last version of DexiNed is implemented in Pytorch, in TF2 it 
+will be available soon. If to want our preliminary version, please check legacy DIR.
+
+We have prepared a basic intro to DexiNed in a video, [check it in YouTube](https://youtu.be/Hz0uU04B3U8)
+
+<div style="text-align:center"><img src='figs/DexiNed_banner.png' width=800>
+</div>
+
+## Table of Contents
+* [DexiNed-v1](#dexiNed-v1)
+* [TensorFlow](#tensorflow2)
+* [Datasets](#datasets)
+* [Performance](#performance)
+* [Citation](#citation)
+
+# DexiNed-v1
+
+This version was presented in WACV2020, this is the base DexiNed model [implemented in TensorFlow 1](https://github.com/xavysp/DexiNed/tree/master/legacy)
+
+
+# TensorFlow2
+
+Dexined version on TF 2 is not ready 
+## Requirements
+
+* [Python 3.7](https://www.python.org/downloads/release/python-370/g)
+* [Pytorch >=1.4](https://pytorch.org/) (Last test 1.9)
+* [OpenCV](https://pypi.org/project/opencv-python/)
+* [Matplotlib](https://matplotlib.org/3.1.1/users/installing.html)
+* [Kornia](https://kornia.github.io/)
+* Other package like Numpy, h5py, PIL, json. 
+
+Once the packages are installed,  clone this repo as follow: 
+
+    git clone https://github.com/xavysp/DexiNed.git
+    cd DexiNed
+
+## Project Architecture
+
+```
+├── data                        # Sample images for testing (paste your image here)
+|   ├── lena_std.tif            # Sample 1
+├── DexiNed-TF2                 # DexiNed in TensorFlow2 (in construction)   
+├── figs                        # Images used in README.md
+|   └── DexiNed_banner.png      # DexiNed banner
+├── legacy                      # DexiNed in TensorFlow1 (presented in WACV2020)
+├── utls                        # A series of tools used in this repo
+|   └── image.py                # Miscellaneous tool functions
+├── datasets.py                 # Tools for dataset managing 
+├── dexi_utils.py               # New functions still not used in the currecnt version
+├── losses.py                   # Loss function used to train DexiNed (BDCNloss2)
+├── main.py                     # The main python file with main functions and parameter settings
+                                # here you can test and train
+├── model.py                    # DexiNed class in pythorch
+```
+
+Before to start please check dataset.py, from the first line of code you can see the datasets used for training/testing. The main.py, line 194, call the data  for the training or testing, see the example of the code below:
+```
+    parser = argparse.ArgumentParser(description='DexiNed trainer.')
+    parser.add_argument('--choose_test_data',
+                        type=int,
+                        default=1,
+                        help='Already set the dataset for testing choice: 0 - 8')
+    # ----------- test -------0--
+
+    TEST_DATA = DATASET_NAMES[parser.parse_args().choose_test_data] # max 8
+    test_inf = dataset_info(TEST_DATA, is_linux=IS_LINUX)
+    test_dir = test_inf['data_dir']
+    is_testing = True# current test -352-SM-NewGT-2AugmenPublish
+
+    # Training settings
+    TRAIN_DATA = DATASET_NAMES[0] # BIPED=0
+    train_inf = dataset_info(TRAIN_DATA, is_linux=IS_LINUX)
+    train_dir = train_inf['data_dir']
+```
+
+## Test
+As previously mentioned, the datasets.py has, among other things, the whole datasets configurations used in DexiNed for testing and training:
+```
+DATASET_NAMES = [
+    'BIPED',
+    'BSDS',
+    'BSDS300',
+    'CID',
+    'DCD',
+    'MULTICUE', #5
+    'PASCAL',
+    'NYUD',
+    'CLASSIC'
+] 
+```
+For example, if want to test your own dataset or image choose "CLASSIC" and save your test data in "data" dir.
+Before test the DexiNed model, it is necesarry to download the checkpoint here [Checkpoint Pytorch](https://drive.google.com/file/d/1V56vGTsu7GYiQouCIKvTWl5UKCZ6yCNu/view?usp=sharing) and save this file into the DexiNed folder like: checkpoints/BIPED/10/(here the checkpoints from Drive), then run as follow:
+
+```python main.py --choose_test_data=-1 ```
+Make sure that in main.py the test setting be as:
+```parser.add_argument('--is_testing', default=True, help='Script in testing mode.')```
+DexiNed downsample the input image till 16 scales, please make sure that, in dataset_info fucn (datasets.py), the image width and height be multiple of 16, like 512, 960, and etc. **In the Checkpoint from Drive you will find the last trained checkpoint, which has been trained in the last version of BIPED dataset that will be updated soon in Kaggle **
+
+## Train
+
+    python main.py 
+Make sure that in main.py the train setting be as:
+```parser.add_argument('--is_testing', default=False, help='Script in testing mode.')```
+
+# Datasets
+
+## Dataset used for Training
+```diff
+- The BIPED dataset has been updated to the last version and it can be download from Kaggle
+
+```
+
+BIPED (Barcelona Images for Perceptual Edge Detection): This dataset is collected and annotated in the edge level for this work. **See more details, augmentation script, and download** in: [Option1](https://xavysp.github.io/MBIPED/), [Option2 kaggle](https://www.kaggle.com/xavysp/biped). The BIPED dataset has been updated, adding more annotations and correcting few mistakes, so those links have the renewed version of BIPED, if you want the older version you may ask us by email. The last performance (table below) will be updated soon. 
+
+## Datasets used for Testing
+
+Edge detection datasets
+* [BIPED](https://xavysp.github.io/MBIPED/) and [MDBD](http://serre-lab.clps.brown.edu/resource/multicue/)
+
+Non-edge detection datasets
+
+* [CID](http://www.cs.rug.nl/~imaging/databases/contour_database/contour_database.html) <!-- * [DCD](http://www.cs.cmu.edu/~mengtial/proj/sketch/)-->, [BSDS300](https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/), [BSDS500](https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/), [NYUD](https://cs.nyu.edu/~silberman/datasets/nyu_depth_v2.html), and [PASCAL-context](https://cs.stanford.edu/~roozbeh/pascal-context/)
+
+# Performance
+
+The results below are from the last version of BIPEP (BIPEDv2, we leave as BIPED because this is the default version).
+After WACV20, the BIPED images have been checked again and added more annotations. All of those models have been trained again. 
+
+<center>
+
+|     Methods    |    ODS   |    ODS   |    AP    |
+| -------------- | ---------| -------- | -------- |
+| [RCF](https://github.com/yun-liu/rcf)      | `.849` | `.861` | `.906` |
+| [BDCN](https://github.com/pkuCactus/BDCN)     | `.890` | `.899` | `.934` |
+| [CATS](https://github.com/WHUHLX/CATS)     | `.887` | `.892` | `.817` |
+| DexiNed-f(Ours)| `.895` | `.900` | `.927` |
+| DexiNed-a(Ours)| `.893` | `.897` | `.940` |
+
+</center>
+
+
+# Citation
+
+If you like DexiNed, why not starring the project on GitHub!
+
+[![GitHub stars](https://img.shields.io/github/stars/xavysp/DexiNed.svg?style=social&label=Star&maxAge=3600)](https://GitHub.com/xavysp/DexiNed/stargazers/)
+
+Please cite our paper if you find helpful in your academic/scientific publication,
+```
+@INPROCEEDINGS {xsoria2020dexined,
+author = {X. Soria and E. Riba and A. Sappa},
+booktitle = {2020 IEEE Winter Conference on Applications of Computer Vision (WACV)},
+title = {Dense Extreme Inception Network: Towards a Robust CNN Model for Edge Detection},
+year = {2020},
+volume = {},
+issn = {},
+pages = {1912-1921},
+keywords = {image edge detection;convolution;training;feeds;machine learning;task analysis;kernel},
+doi = {10.1109/WACV45572.2020.9093290},
+url = {https://doi.ieeecomputersociety.org/10.1109/WACV45572.2020.9093290},
+publisher = {IEEE Computer Society},
+address = {Los Alamitos, CA, USA},
+month = {mar}
+}
+```
+Extended version.
+```
+@article{soria2023dexined_ext,
+title = {Dense extreme inception network for edge detection},
+journal = {Pattern Recognition},
+volume = {139},
+pages = {109461},
+year = {2023},
+issn = {0031-3203},
+doi = {https://doi.org/10.1016/j.patcog.2023.109461},
+url = {https://www.sciencedirect.com/science/article/pii/S0031320323001619},
+author = {Xavier Soria and Angel Sappa and Patricio Humanante and Arash Akbarinia},
+keywords = {Edge detection, Deep learning, CNN, Contour detection, Boundary detection, Segmentation}
+}
+```
+

app.py

@@ -0,0 +1,84 @@
+import os
+import glob
+import subprocess
+import streamlit as st
+from PIL import Image
+
+# ————————————————————————————————
+# 1) Setup direktori dan session state
+# ————————————————————————————————
+DATA_DIR = "data"
+FUSED_DIR = os.path.join("result", "BIPED2CLASSIC", "fused")
+
+os.makedirs(DATA_DIR, exist_ok=True)
+os.makedirs(FUSED_DIR, exist_ok=True)
+
+# Inisialisasi flag supaya hasil hanya tampil setelah proses
+if "processed" not in st.session_state:
+    st.session_state.processed = False
+
+st.title("Deteksi jalan berlubang menggunakan DexiNed Egde Detection")
+
+# ————————————————————————————————
+# 2) Upload input
+# ————————————————————————————————
+upload = st.file_uploader("Unggah satu gambar jalan", type=["jpg","jpeg","png"])
+if upload:
+    st.session_state.uploaded_file = upload
+else:
+    # reset kalau user hapus input
+    st.session_state.processed = False
+
+# ————————————————————————————————
+# 3) Tombol Proses
+# ————————————————————————————————
+if st.button("Memproses Gambar"):
+    
+    # 3.0 Validasi apakah gambar sudah diupload
+    if not upload and "uploaded_file" not in st.session_state:
+        st.error("❌ Silakan upload gambar terlebih dahulu sebelum memproses!")
+    elif not upload and st.session_state.get("uploaded_file") is None:
+        st.error("❌ Silakan upload gambar terlebih dahulu sebelum memproses!")
+    else:
+        # 3.1 Hapus semua file lama di DATA_DIR
+        for f in glob.glob(os.path.join(DATA_DIR, "*")):
+            os.remove(f)
+
+        # 3.2 Simpan ulang file input (seharusnya hanya 1)
+        input_path = os.path.join(DATA_DIR, st.session_state.uploaded_file.name)
+        with open(input_path, "wb") as f:
+            f.write(st.session_state.uploaded_file.getvalue())
+
+        # 3.3 Hapus semua file lama di FUSED_DIR
+        for f in glob.glob(os.path.join(FUSED_DIR, "*")):
+            os.remove(f)
+
+        # 3.4 Jalankan main.py (sesuaikan args jika perlu)
+        with st.spinner("Sedang memproses gambar..."):
+            result = subprocess.run(
+                ["python", "main.py"],
+                capture_output=True, text=True
+            )
+        
+        if result.returncode == 0:
+            st.success("Proses selesai 🎉")
+            st.session_state.processed = True
+        else:
+            st.error("Proses gagal:")
+            st.code(result.stderr)
+            st.session_state.processed = False
+
+# ————————————————————————————————
+# 4) Tampilkan hasil jika sudah diproses
+# ————————————————————————————————
+if st.session_state.processed:
+    # Cari file di fused (harusnya 1 file output)
+    outputs = glob.glob(os.path.join(FUSED_DIR, "*.*"))
+    if len(outputs) == 1:
+        out_img = Image.open(outputs[0])
+        st.subheader("Gambar Hasil")
+        st.image(out_img, caption=os.path.basename(outputs[0]), use_container_width=True)
+    elif len(outputs) > 1:
+        st.warning(f"Ditemukan {len(outputs)} file, seharusnya hanya 1.")
+    else:
+        st.error("Tidak ada file output di folder fused.")

checkpoints/BIPED/10/10_model.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd4c603ef71113b447bffb72a73b9a54ca890e78f4bc8f34cb73f1efb2681e74
+size 141069187

dataset-lists/MDBD/test_pair.lst

@@ -0,0 +1,20 @@
+test/imgs/4100_left_2268.png test/edge-maps/4100_left_2268.png
+test/imgs/4101_left_0382.png test/edge-maps/4101_left_0382.png
+test/imgs/4108_left_0082.png test/edge-maps/4108_left_0082.png
+test/imgs/4113_left_0082.png test/edge-maps/4113_left_0082.png
+test/imgs/4114_left_0611.png test/edge-maps/4114_left_0611.png
+test/imgs/4176_left_0273.png test/edge-maps/4176_left_0273.png
+test/imgs/4178_left_0052.png test/edge-maps/4178_left_0052.png
+test/imgs/4179_left_0315.png test/edge-maps/4179_left_0315.png
+test/imgs/4187_left_0071.png test/edge-maps/4187_left_0071.png
+test/imgs/4209_left_0034.png test/edge-maps/4209_left_0034.png
+test/imgs/4377_left_0016.png test/edge-maps/4377_left_0016.png
+test/imgs/4386_left_0020.png test/edge-maps/4386_left_0020.png
+test/imgs/4406_left_0030.png test/edge-maps/4406_left_0030.png
+test/imgs/4523_left_0070.png test/edge-maps/4523_left_0070.png
+test/imgs/4558_left_0260.png test/edge-maps/4558_left_0260.png
+test/imgs/4580_left_0165.png test/edge-maps/4580_left_0165.png
+test/imgs/4603_left_0050.png test/edge-maps/4603_left_0050.png
+test/imgs/4663_left_0010.png test/edge-maps/4663_left_0010.png
+test/imgs/4684_left_0010.png test/edge-maps/4684_left_0010.png
+test/imgs/4685_left_0045.png test/edge-maps/4685_left_0045.png

datasets.py

@@ -0,0 +1,490 @@
+import os
+import random
+
+import cv2
+import numpy as np
+import torch
+from torch.utils.data import Dataset
+import json
+
+DATASET_NAMES = [
+    'BIPED',
+    'BSDS',
+    'BRIND',
+    'BSDS300',
+    'CID',
+    'DCD',
+    'MDBD', #5
+    'PASCAL',
+    'NYUD',
+    'CLASSIC'
+]  # 8
+
+
+def dataset_info(dataset_name, is_linux=True):
+    if is_linux:
+
+        config = {
+            'BSDS': {
+                'img_height': 512, #321
+                'img_width': 512, #481
+                'train_list': 'train_pair.lst',
+                'test_list': 'test_pair.lst',
+                'data_dir': '/opt/dataset/BSDS',  # mean_rgb
+                'yita': 0.5
+            },
+            'BRIND': {
+                'img_height': 512,  # 321
+                'img_width': 512,  # 481
+                'train_list': 'train_pair2.lst',
+                'test_list': 'test_pair.lst',
+                'data_dir': '/opt/dataset/BRIND',  # mean_rgb
+                'yita': 0.5
+            },
+            'BSDS300': {
+                'img_height': 512, #321
+                'img_width': 512, #481
+                'test_list': 'test_pair.lst',
+                'train_list': None,
+                'data_dir': '/opt/dataset/BSDS300',  # NIR
+                'yita': 0.5
+            },
+            'PASCAL': {
+                'img_height': 416, # 375
+                'img_width': 512, #500
+                'test_list': 'test_pair.lst',
+                'train_list': None,
+                'data_dir': '/opt/dataset/PASCAL',  # mean_rgb
+                'yita': 0.3
+            },
+            'CID': {
+                'img_height': 512,
+                'img_width': 512,
+                'test_list': 'test_pair.lst',
+                'train_list': None,
+                'data_dir': '/opt/dataset/CID',  # mean_rgb
+                'yita': 0.3
+            },
+            'NYUD': {
+                'img_height': 448,#425
+                'img_width': 560,#560
+                'test_list': 'test_pair.lst',
+                'train_list': None,
+                'data_dir': '/opt/dataset/NYUD',  # mean_rgb
+                'yita': 0.5
+            },
+            'MDBD': {
+                'img_height': 720,
+                'img_width': 1280,
+                'test_list': 'test_pair.lst',
+                'train_list': 'train_pair.lst',
+                'data_dir': '/opt/dataset/MDBD',  # mean_rgb
+                'yita': 0.3
+            },
+            'BIPED': {
+                'img_height': 720, #720 # 1088
+                'img_width': 1280, # 1280 5 1920
+                'test_list': 'test_pair.lst',
+                'train_list': 'train_rgb.lst',
+                'data_dir': '/opt/dataset/BIPED',  # mean_rgb
+                'yita': 0.5
+            },
+            'CLASSIC': {
+                'img_height': 512,
+                'img_width': 512,
+                'test_list': None,
+                'train_list': None,
+                'data_dir': 'data',  # mean_rgb
+                'yita': 0.5
+            },
+            'DCD': {
+                'img_height': 352, #240
+                'img_width': 480,# 360
+                'test_list': 'test_pair.lst',
+                'train_list': None,
+                'data_dir': '/opt/dataset/DCD',  # mean_rgb
+                'yita': 0.2
+            }
+        }
+    else:
+        config = {
+            'BSDS': {'img_height': 512,  # 321
+                     'img_width': 512,  # 481
+                     'test_list': 'test_pair.lst',
+                     'train_list': 'train_pair.lst',
+                     'data_dir': 'C:/Users/xavysp/dataset/BSDS',  # mean_rgb
+                     'yita': 0.5},
+            'BSDS300': {'img_height': 512,  # 321
+                        'img_width': 512,  # 481
+                        'test_list': 'test_pair.lst',
+                        'data_dir': 'C:/Users/xavysp/dataset/BSDS300',  # NIR
+                        'yita': 0.5},
+            'PASCAL': {'img_height': 375,
+                       'img_width': 500,
+                       'test_list': 'test_pair.lst',
+                       'data_dir': 'C:/Users/xavysp/dataset/PASCAL',  # mean_rgb
+                       'yita': 0.3},
+            'CID': {'img_height': 512,
+                    'img_width': 512,
+                    'test_list': 'test_pair.lst',
+                    'data_dir': 'C:/Users/xavysp/dataset/CID',  # mean_rgb
+                    'yita': 0.3},
+            'NYUD': {'img_height': 425,
+                     'img_width': 560,
+                     'test_list': 'test_pair.lst',
+                     'data_dir': 'C:/Users/xavysp/dataset/NYUD',  # mean_rgb
+                     'yita': 0.5},
+            'MDBD': {'img_height': 720,
+                         'img_width': 1280,
+                         'test_list': 'test_pair.lst',
+                         'train_list': 'train_pair.lst',
+                         'data_dir': 'C:/Users/xavysp/dataset/MDBD',  # mean_rgb
+                         'yita': 0.3},
+            'BIPED': {'img_height': 720,  # 720
+                      'img_width': 1280,  # 1280
+                      'test_list': 'test_pair.lst',
+                      'train_list': 'train_rgb.lst',
+                      'data_dir': 'C:/Users/xavysp/dataset/BIPED',  # WIN: '../.../dataset/BIPED/edges'
+                      'yita': 0.5},
+            'CLASSIC': {'img_height': 512,
+                        'img_width': 512,
+                        'test_list': None,
+                        'train_list': None,
+                        'data_dir': 'data',  # mean_rgb
+                        'yita': 0.5},
+            'DCD': {'img_height': 240,
+                    'img_width': 360,
+                    'test_list': 'test_pair.lst',
+                    'data_dir': 'C:/Users/xavysp/dataset/DCD',  # mean_rgb
+                    'yita': 0.2}
+        }
+    return config[dataset_name]
+
+
+class TestDataset(Dataset):
+    def __init__(self,
+                 data_root,
+                 test_data,
+                 mean_bgr,
+                 img_height,
+                 img_width,
+                 test_list=None,
+                 arg=None
+                 ):
+        if test_data not in DATASET_NAMES:
+            raise ValueError(f"Unsupported dataset: {test_data}")
+
+        self.data_root = data_root
+        self.test_data = test_data
+        self.test_list = test_list
+        self.args=arg
+        # self.arg = arg
+        # self.mean_bgr = arg.mean_pixel_values[0:3] if len(arg.mean_pixel_values) == 4 \
+        #     else arg.mean_pixel_values
+        self.mean_bgr = mean_bgr
+        self.img_height = img_height
+        self.img_width = img_width
+        self.data_index = self._build_index()
+
+        print(f"mean_bgr: {self.mean_bgr}")
+
+    def _build_index(self):
+        sample_indices = []
+        if self.test_data == "CLASSIC":
+            # for single image testing
+            images_path = os.listdir(self.data_root)
+            labels_path = None
+            sample_indices = [images_path, labels_path]
+        else:
+            # image and label paths are located in a list file
+
+            if not self.test_list:
+                raise ValueError(
+                    f"Test list not provided for dataset: {self.test_data}")
+
+            list_name = os.path.join(self.data_root, self.test_list)
+            if self.test_data.upper()=='BIPED':
+
+                with open(list_name) as f:
+                    files = json.load(f)
+                for pair in files:
+                    tmp_img = pair[0]
+                    tmp_gt = pair[1]
+                    sample_indices.append(
+                        (os.path.join(self.data_root, tmp_img),
+                         os.path.join(self.data_root, tmp_gt),))
+            else:
+                with open(list_name, 'r') as f:
+                    files = f.readlines()
+                files = [line.strip() for line in files]
+                pairs = [line.split() for line in files]
+
+                for pair in pairs:
+                    tmp_img = pair[0]
+                    tmp_gt = pair[1]
+                    sample_indices.append(
+                        (os.path.join(self.data_root, tmp_img),
+                         os.path.join(self.data_root, tmp_gt),))
+        return sample_indices
+
+    def __len__(self):
+        return len(self.data_index[0]) if self.test_data.upper()=='CLASSIC' else len(self.data_index)
+
+    def __getitem__(self, idx):
+        # get data sample
+        # image_path, label_path = self.data_index[idx]
+        if self.data_index[1] is None:
+            image_path = self.data_index[0][idx]
+        else:
+            image_path = self.data_index[idx][0]
+        label_path = None if self.test_data == "CLASSIC" else self.data_index[idx][1]
+        img_name = os.path.basename(image_path)
+        file_name = os.path.splitext(img_name)[0] + ".png"
+
+        # base dir
+        if self.test_data.upper() == 'BIPED':
+            img_dir = os.path.join(self.data_root, 'imgs', 'test')
+            gt_dir = os.path.join(self.data_root, 'edge_maps', 'test')
+        elif self.test_data.upper() == 'CLASSIC':
+            img_dir = self.data_root
+            gt_dir = None
+        else:
+            img_dir = self.data_root
+            gt_dir = self.data_root
+
+        # load data
+        image = cv2.imread(os.path.join(img_dir, image_path), cv2.IMREAD_COLOR)
+        if not self.test_data == "CLASSIC":
+            label = cv2.imread(os.path.join(
+                gt_dir, label_path), cv2.IMREAD_COLOR)
+        else:
+            label = None
+
+        im_shape = [image.shape[0], image.shape[1]]
+        image, label = self.transform(img=image, gt=label)
+
+        return dict(images=image, labels=label, file_names=file_name, image_shape=im_shape)
+
+    def transform(self, img, gt):
+        # gt[gt< 51] = 0 # test without gt discrimination
+        if self.test_data == "CLASSIC":
+            img_height = self.img_height
+            img_width = self.img_width
+            print(
+                f"actual size: {img.shape}, target size: {( img_height,img_width,)}")
+            # img = cv2.resize(img, (self.img_width, self.img_height))
+            img = cv2.resize(img, (img_width,img_height))
+            gt = None
+
+        # Make images and labels at least 512 by 512
+        elif img.shape[0] < 512 or img.shape[1] < 512:
+            img = cv2.resize(img, (self.args.test_img_width, self.args.test_img_height)) # 512
+            gt = cv2.resize(gt, (self.args.test_img_width, self.args.test_img_height)) # 512
+
+        # Make sure images and labels are divisible by 2^4=16
+        elif img.shape[0] % 16 != 0 or img.shape[1] % 16 != 0:
+            img_width = ((img.shape[1] // 16) + 1) * 16
+            img_height = ((img.shape[0] // 16) + 1) * 16
+            img = cv2.resize(img, (img_width, img_height))
+            gt = cv2.resize(gt, (img_width, img_height))
+        else:
+            img_width =self.args.test_img_width
+            img_height =self.args.test_img_height
+            img = cv2.resize(img, (img_width, img_height))
+            gt = cv2.resize(gt, (img_width, img_height))
+
+        # if self.yita is not None:
+        #     gt[gt >= self.yita] = 1
+        img = np.array(img, dtype=np.float32)
+        # if self.rgb:
+        #     img = img[:, :, ::-1]  # RGB->BGR
+        # img=cv2.resize(img, (400, 464))
+        img -= self.mean_bgr
+        img = img.transpose((2, 0, 1))
+        img = torch.from_numpy(img.copy()).float()
+
+        if self.test_data == "CLASSIC":
+            gt = np.zeros((img.shape[:2]))
+            gt = torch.from_numpy(np.array([gt])).float()
+        else:
+            gt = np.array(gt, dtype=np.float32)
+            if len(gt.shape) == 3:
+                gt = gt[:, :, 0]
+            gt /= 255.
+            gt = torch.from_numpy(np.array([gt])).float()
+
+        return img, gt
+
+
+class BipedDataset(Dataset):
+    train_modes = ['train', 'test', ]
+    dataset_types = ['rgbr', ]
+    data_types = ['aug', ]
+
+    def __init__(self,
+                 data_root,
+                 img_height,
+                 img_width,
+                 mean_bgr,
+                 train_mode='train',
+                 dataset_type='rgbr',
+                 #  is_scaling=None,
+                 # Whether to crop image or otherwise resize image to match image height and width.
+                 crop_img=False,
+                 arg=None
+                 ):
+        self.data_root = data_root
+        self.train_mode = train_mode
+        self.dataset_type = dataset_type
+        self.data_type = 'aug'  # be aware that this might change in the future
+        self.img_height = img_height
+        self.img_width = img_width
+        self.mean_bgr = mean_bgr
+        self.crop_img = crop_img
+        self.arg = arg
+
+        self.data_index = self._build_index()
+
+    def _build_index(self):
+        assert self.train_mode in self.train_modes, self.train_mode
+        assert self.dataset_type in self.dataset_types, self.dataset_type
+        assert self.data_type in self.data_types, self.data_type
+
+        data_root = os.path.abspath(self.data_root)
+        sample_indices = []
+        if self.arg.train_data.lower()=='biped':
+
+            images_path = os.path.join(data_root,
+                                       'edges/imgs',
+                                       self.train_mode,
+                                       self.dataset_type,
+                                       self.data_type)
+            labels_path = os.path.join(data_root,
+                                       'edges/edge_maps',
+                                       self.train_mode,
+                                       self.dataset_type,
+                                       self.data_type)
+
+            for directory_name in os.listdir(images_path):
+                image_directories = os.path.join(images_path, directory_name)
+                for file_name_ext in os.listdir(image_directories):
+                    file_name = os.path.splitext(file_name_ext)[0]
+                    sample_indices.append(
+                        (os.path.join(images_path, directory_name, file_name + '.jpg'),
+                         os.path.join(labels_path, directory_name, file_name + '.png'),)
+                    )
+        else:
+            file_path = os.path.join(data_root, self.arg.train_list)
+            if self.arg.train_data.lower()=='bsds':
+
+                with open(file_path, 'r') as f:
+                    files = f.readlines()
+                files = [line.strip() for line in files]
+
+                pairs = [line.split() for line in files]
+                for pair in pairs:
+                    tmp_img = pair[0]
+                    tmp_gt = pair[1]
+                    sample_indices.append(
+                        (os.path.join(data_root,tmp_img),
+                         os.path.join(data_root,tmp_gt),))
+            else:
+                with open(file_path) as f:
+                    files = json.load(f)
+                for pair in files:
+                    tmp_img = pair[0]
+                    tmp_gt = pair[1]
+                    sample_indices.append(
+                        (os.path.join(data_root, tmp_img),
+                         os.path.join(data_root, tmp_gt),))
+
+        return sample_indices
+
+    def __len__(self):
+        return len(self.data_index)
+
+    def __getitem__(self, idx):
+        # get data sample
+        image_path, label_path = self.data_index[idx]
+
+        # load data
+        image = cv2.imread(image_path, cv2.IMREAD_COLOR)
+        label = cv2.imread(label_path, cv2.IMREAD_GRAYSCALE)
+        image, label = self.transform(img=image, gt=label)
+        return dict(images=image, labels=label)
+
+    def transform(self, img, gt):
+        gt = np.array(gt, dtype=np.float32)
+        if len(gt.shape) == 3:
+            gt = gt[:, :, 0]
+
+        gt /= 255. # for DexiNed input and BDCN
+
+        img = np.array(img, dtype=np.float32)
+        img -= self.mean_bgr
+        i_h, i_w,_ = img.shape
+        # data = []
+        # if self.scale is not None:
+        #     for scl in self.scale:
+        #         img_scale = cv2.resize(img, None, fx=scl, fy=scl, interpolation=cv2.INTER_LINEAR)
+        #         data.append(torch.from_numpy(img_scale.transpose((2, 0, 1))).float())
+        #     return data, gt
+        #  400 for BIPEd and 352 for BSDS check with 384
+        crop_size = self.img_height if self.img_height == self.img_width else None#448# MDBD=480 BIPED=480/400 BSDS=352
+
+        # for BSDS 352/BRIND
+        if i_w> crop_size and i_h>crop_size:
+            i = random.randint(0, i_h - crop_size)
+            j = random.randint(0, i_w - crop_size)
+            img = img[i:i + crop_size , j:j + crop_size ]
+            gt = gt[i:i + crop_size , j:j + crop_size ]
+
+        # # for BIPED/MDBD
+        # if np.random.random() > 0.4: #l
+        #     h,w = gt.shape
+        #     if i_w> 500 and i_h>500:
+        #
+        #         LR_img_size = crop_size #l BIPED=256, 240 200 # MDBD= 352 BSDS= 176
+        #         i = random.randint(0, h - LR_img_size)
+        #         j = random.randint(0, w - LR_img_size)
+        #         # if img.
+        #         img = img[i:i + LR_img_size , j:j + LR_img_size ]
+        #         gt = gt[i:i + LR_img_size , j:j + LR_img_size ]
+        #     else:
+        #         LR_img_size = 352#256  # l BIPED=208-352, # MDBD= 352-480- BSDS= 176-320
+        #         i = random.randint(0, h - LR_img_size)
+        #         j = random.randint(0, w - LR_img_size)
+        #         # if img.
+        #         img = img[i:i + LR_img_size, j:j + LR_img_size]
+        #         gt = gt[i:i + LR_img_size, j:j + LR_img_size]
+        #         img = cv2.resize(img, dsize=(crop_size, crop_size), )
+        #         gt = cv2.resize(gt, dsize=(crop_size, crop_size))
+
+        else:
+            # New addidings
+            img = cv2.resize(img, dsize=(crop_size, crop_size))
+            gt = cv2.resize(gt, dsize=(crop_size, crop_size))
+        # BRIND
+        gt[gt > 0.1] +=0.2#0.4
+        gt = np.clip(gt, 0., 1.)
+        # gt[gt > 0.1] =1#0.4
+        # gt = np.clip(gt, 0., 1.)
+        # # for BIPED
+        # gt[gt > 0.2] += 0.6# 0.5 for BIPED
+        # gt = np.clip(gt, 0., 1.) # BIPED
+        # # for MDBD
+        # gt[gt > 0.1] +=0.7
+        # gt = np.clip(gt, 0., 1.)
+        # # For RCF input
+        # # -----------------------------------
+        # gt[gt==0]=0.
+        # gt[np.logical_and(gt>0.,gt<0.5)] = 2.
+        # gt[gt>=0.5]=1.
+        #
+        # gt = gt.astype('float32')
+        # ----------------------------------
+
+        img = img.transpose((2, 0, 1))
+        img = torch.from_numpy(img.copy()).float()
+        gt = torch.from_numpy(np.array([gt])).float()
+        return img, gt

dexi_utils.py

@@ -0,0 +1,87 @@
+import cv2 as cv
+import numpy as np
+import torch
+
+
+def image_normalization(img, img_min=0, img_max=255):
+    """This is a typical image normalization function
+    where the minimum and maximum of the image is needed
+    source: https://en.wikipedia.org/wiki/Normalization_(image_processing)
+    :param img: an image could be gray scale or color
+    :param img_min:  for default is 0
+    :param img_max: for default is 255
+    :return: a normalized image, if max is 255 the dtype is uint8
+    """
+    img = np.float32(img)
+    epsilon = 1e-12  # whenever an inconsistent image
+    img = (img-np.min(img))*(img_max-img_min) / \
+        ((np.max(img)-np.min(img))+epsilon)+img_min
+    return img
+
+
+# def visualize_result(imgs_list, arg):
+#     """
+#     function for tensorflow results
+#     :param imgs_list: a list of prediction, gt and input data
+#     :param arg:
+#     :return: one image with the whole of imgs_list data
+#     """
+#     n_imgs = len(imgs_list)
+#     data_list = []
+#     for i in range(n_imgs):
+#         tmp = imgs_list[i]
+
+#         if tmp.shape[1] == 3:
+#             tmp = np.transpose(np.squeeze(tmp), [1, 2, 0])
+#             tmp = restore_rgb(
+#                 [arg.channel_swap, arg.mean_pixel_values[:3]], tmp)
+#             tmp = np.uint8(image_normalization(tmp))
+#         else:
+#             tmp = np.squeeze(tmp)
+#             if len(tmp.shape) == 2:
+#                 tmp = np.uint8(image_normalization(tmp))
+#                 tmp = cv.bitwise_not(tmp)
+#                 tmp = cv.cvtColor(tmp, cv.COLOR_GRAY2BGR)
+#             else:
+#                 tmp = np.uint8(image_normalization(tmp))
+#         data_list.append(tmp)
+#     img = data_list[0]
+#     if n_imgs % 2 == 0:
+#         imgs = np.zeros((img.shape[0] * 2 + 10, img.shape[1]
+#                          * (n_imgs // 2) + ((n_imgs // 2 - 1) * 5), 3))
+#     else:
+#         imgs = np.zeros((img.shape[0] * 2 + 10, img.shape[1]
+#                          * ((1 + n_imgs) // 2) + ((n_imgs // 2) * 5), 3))
+#         n_imgs += 1
+
+#     k = 0
+#     imgs = np.uint8(imgs)
+#     i_step = img.shape[0]+10
+#     j_step = img.shape[1]+5
+#     for i in range(2):
+#         for j in range(n_imgs//2):
+#             if k < len(data_list):
+#                 imgs[i*i_step:i*i_step+img.shape[0], j*j_step:j *
+#                      j_step+img.shape[1], :] = data_list[k]
+#                 k += 1
+#             else:
+#                 pass
+#     return imgs
+
+
+# def cv_imshow(title='image', img=None):
+#     cv.imshow(title, img)
+#     cv.waitKey(0)
+#     cv.destroyAllWindows()
+def tensor2edge(tensor):
+    print(tensor.shape)
+    tensor =torch.squeeze(tensor) if len(tensor.shape)>2 else tensor
+    tmp = torch.sigmoid(tensor)
+    tmp = tmp.cpu().detach().numpy()
+    # tmp = np.transpose(np.squeeze(tmp[1]), [1, 2, 0])
+    tmp = np.uint8(image_normalization(tmp))
+    tmp = cv.bitwise_not(tmp)
+    tmp = cv.cvtColor(tmp, cv.COLOR_GRAY2BGR)
+    cv.imshow('test_img', tmp)
+    cv.waitKey(0)
+    cv.destroyAllWindows()

legacy/README.md

@@ -0,0 +1,132 @@
+[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/dense-extreme-inception-network-towards-a/edge-detection-on-biped)](https://paperswithcode.com/sota/edge-detection-on-biped?p=dense-extreme-inception-network-towards-a)
+
+# Dense Extreme Inception Network: Towards a Robust CNN Model for Edge Detection (DexiNed)
+
+<!-- ```diff
+- Sorry for any inconvenience, we are updating the repo
+``` -->
+
+This work presents a new Convolutional Neural Network (CNN) arquitecture for edge detection. Unlike of the state-of-the-art CNN based edge detectors, this models has a single training stage, but it is still able to overcome those models in the edge detection datasets. Moreover, Dexined does not need pre-trained weights, and it is trained from the scratch with fewer parameters tunning. To know more about DexiNed, read our first version of Dexined in [arxiv](https://arxiv.org/abs/1909.01955), the last version will be uploaded after the camera-ready deadline of WACV2020.
+
+<div style="text-align:center"><img src='figs/DexiNed_banner.png' width=800>
+<div/>
+
+## Table of Contents
+* [Datasets](#datasets)
+* [Performance](#performance)
+* [Citation](#citation)
+
+This is the first version of DexiNed presented in WACV2020, so it can run on TensorFlow 1.15.4. The last version of DexiNed has been implemented on Pytorch and you can see in the main path of this repo.
+## Requirements
+
+* [Python 3.7](https://www.python.org/downloads/release/python-370/g)
+* [TensorFlow>=1.8 <=1.15.4](https://www.tensorflow.org) (tested on such versions)
+* [OpenCV](https://pypi.org/project/opencv-python/)
+* [Matplotlib](https://matplotlib.org/3.1.1/users/installing.html)
+* Other package like Numpy, h5py, PIL. 
+
+Once the packages are installed,  clone this repo as follow: 
+
+    git clone https://github.com/xavysp/DexiNed.git
+    cd DexiNed
+
+## Project Architecture
+
+```
+├── data                        # sample images for testing
+|   ├── lena_std.tif            # sample 1
+|   └── stonehengeuk.jpg        # sample 2
+├── figs                        # Images used in README.md
+|   └── DexiNed_banner.png      # DexiNed banner
+├── models                      # tensorflow model file  
+|   └── dexined.py              # DexiNed class
+├── utls                        # a series of tools used in this repo
+|   └── dataset_manager.py      # tools for dataset managing
+|   └── losses.py               # Loss function used to train DexiNed 
+|   └── utls.py                 # miscellaneous tool functions
+├── run_model.py                # the main python file with main functions and parameter settings
+└── test.py                     # the script to run the test experiment
+└── train.py                    # the script to run the train experiment
+```
+
+As described above, run_model.py has the parameters settings, whether DexiNed is used for training or testing, before those processes the parameters need to be set. As highlighted, DexiNed is trained just one time with our proposed dataset BIPED, so in "--train_dataset" as the default setting is BIDEP; however, in the testing stage (--test_dataset), any dataset can be used, even CLASSIC, which is an arbitrary image downloaded from the internet. However, to evaluate with single images or CLASSIC "--use_dataset" has to be in FALSE mode. Whenever a dataset is used to test or train on DexiNed the arguments have to have the list of training or testing files (--train_list, --test_list). Pay attention in the parameters' settings, and change whatever you want, like ''--image_width'' or ''--image_height''. To test the Lena image I set 512x51 (see "test" section).
+```
+parser.add_argument('--train_dataset', default='BIPED', choices=['BIPED','BSDS'])
+parser.add_argument('--test_dataset', default='CLASSIC', choices=['BIPED', 'BSDS','MULTICUE','NYUD','PASCAL','CID'])
+parser.add_argument('--dataset_dir',default=None,type=str)
+parser.add_argument('--dataset_augmented', default=True,type=bool)
+parser.add_argument('--train_list',default='train_rgb.lst', type=str)
+parser.add_argument('--test_list', default='test_pair.lst',type=str)  
+```
+
+## Test
+Before test the DexiNed model, it is necesarry to download the checkpoint here [Checkpoint from Drive](https://drive.google.com/open?id=1fLBpOrSXC2VOWUvDtNGyrHcuB2IB-4_D) and save those files into the DexiNed folder like: checkpoints/DXN_BIPED/train/(here the checkpoints from Drive), then run as follow:
+
+    python run_model.py --image_width=512 --image_height=512
+Make sure that in run_model.py the test setting be as:
+```parser.add_argument('--model_state', default='test', choices=['train','test','None'])```
+DexiNed downsample the input image till 16 scales, please make sure that the image width and height be multiple of 16, like 512, 960, and etc. **In the Checkpoint from Drive you will get data_list.zip, train_1.zip, and train_2.zip. The train_2  contains our last checkpoint trained with the updated BIPED; train_1 has checkpoints with the results presented in WACV'20, and data_list has a list of MDBD dataset images used for testing, if you choose another random list of images, you probably get a better or worst result, I think is not fair.**
+
+## Train
+
+    python run_model.py 
+Make sure that in run_model.py the train setting be as:
+```parser.add_argument('--model_state', default='train', choices=['train','test','None'])```
+
+# Datasets
+
+## Dataset used for Training
+
+BIPED (Barcelona Images for Perceptual Edge Detection): This dataset is collected and annotated in the edge level for this work. See more details and download in: [Option1](https://xavysp.github.io/MBIPED/), [Option2 kaggle](https://www.kaggle.com/xavysp/biped). The BIPED dataset has been updated, adding more annotations and correcting few mistakes, so those links have the renewed version of BIPED, if you want the older version you may ask us by email. The last performance (table below) will be updated soon. 
+
+## Datasets used for Testing
+
+Edge detection datasets
+* [BIPED](https://xavysp.github.io/MBIPED/) and [MDBD](http://serre-lab.clps.brown.edu/resource/multicue/)
+
+Non-edge detection datasets
+
+* [CID](http://www.cs.rug.nl/~imaging/databases/contour_database/contour_database.html) <!-- * [DCD](http://www.cs.cmu.edu/~mengtial/proj/sketch/)-->, [BSDS300](https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/), [BSDS500](https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/), [NYUD](https://cs.nyu.edu/~silberman/datasets/nyu_depth_v2.html), and [PASCAL-context](https://cs.stanford.edu/~roozbeh/pascal-context/)
+
+# Performance
+
+The results below are from the last version of BIPEP. After WACV20, the BIPED images have been again checked and added annotations. All of those models have been trained again. 
+
+<center>
+
+|     Methods    |    ODS   |    ODS   |    AP    |
+| -------------- | ---------| -------- | -------- |
+| [SED](https://github.com/ArashAkbarinia/BoundaryDetection)     | `.717` | `.731` | `.756` |
+| [HED](https://github.com/s9xie/hed)     | `.823` | `.847` | `.869` |
+| [RCF](https://github.com/yun-liu/rcf)     | `.843` | `.859` | `.882` |
+| [BDCN](https://github.com/pkuCactus/BDCN)    | `.839` | `.854` | `.887` |
+| DexiNed(WACV'20)| `.859` | `.867` | `.905` |
+</center>
+Evaluation performed to BIPED dataset. We will update the result soon.
+
+# Citation
+
+If you like DexiNed, why not starring the project on GitHub!
+
+[![GitHub stars](https://img.shields.io/github/stars/xavysp/DexiNed.svg?style=social&label=Star&maxAge=3600)](https://GitHub.com/xavysp/DexiNed/stargazers/)
+
+Please cite our paper if you find helpful in your academic/scientific publication,
+```
+@InProceedings{soria2020dexined,
+    title={Dense Extreme Inception Network: Towards a Robust CNN Model for Edge Detection},
+    author={Xavier Soria and Edgar Riba and Angel Sappa},
+    booktitle={The IEEE Winter Conference on Applications of Computer Vision (WACV '20)},
+    year={2020}
+}
+```
+
+<!--```
+@misc{soria2020dexined_ext,
+    title={Towards a Robust Deep Learning Model for Edge Detection},
+    author={Xavier Soria and Edgar Riba and Angel Sappa},
+    year={2020},
+    eprint={000000000},
+    archivePrefix={arXiv},
+    primaryClass={cs.CV}
+```-->
+

legacy/models/dexined.py

@@ -0,0 +1,503 @@
+""" DexiNed architecture description
+
+
+Created by: Xavier Soria Poma
+Modified from: https://github.com/machrisaa/tensorflow-vgg
+Autonomous University of Barcelona-Computer Vision Center
+[email protected]/[email protected]
+"""
+
+import time
+import os
+import numpy as np
+
+from legacy.utls.losses import *
+from legacy.utls.utls import print_info, print_warning
+
+slim = tf.contrib.slim
+
+
+class dexined():
+
+    def __init__(self, args):
+
+        self.args = args
+        self.utw = self.args.use_trained_model
+        self.img_height =args.image_height
+        self.img_width =args.image_width
+        if args.model_state=='test':
+            self.images = tf.placeholder(tf.float32, [None, self.args.image_height,
+                                                      self.args.image_width, self.args.n_channels])
+        else:
+            self.images = tf.compat.v1.placeholder(tf.float32, [None, self.args.image_height,
+                                                      self.args.image_width, self.args.n_channels])
+        self.edgemaps = tf.compat.v1.placeholder(tf.float32, [None, self.args.image_height,
+                                                    self.args.image_width, 1])
+        self.define_model()
+
+    def define_model(self, is_training=True):
+        """ DexiNed architecture
+        DexiNed is composed by six blocks, the two first blocks have two convolutional layers
+        the rest of the blocks is composed by sub blocks and they have 2, 3, 3, 3 sub blocks
+        """
+        start_time = time.time()
+        use_subpixel=self.args.use_subpixel
+        weight_init =tf.random_normal_initializer(mean=0.0, stddev=0.01)
+        with tf.compat.v1.variable_scope('Xpt') as sc:
+
+            # ------------------------- Block1 ----------------------------------------
+            self.conv1_1 = tf.layers.conv2d(self.images, filters=32, kernel_size=[3, 3],
+                                        strides=(2, 2), bias_initializer=tf.constant_initializer(0.0),
+                                        padding='SAME', name="conv1_1", kernel_initializer=weight_init) #  bx200x200x32, b=batch size
+            self.conv1_1 = slim.batch_norm(self.conv1_1)
+            self.conv1_1 = tf.nn.relu(self.conv1_1)
+
+            self.conv1_2 = tf.layers.conv2d(self.conv1_1, filters=64, kernel_size=[3,3],
+                                        strides=(1,1), bias_initializer=tf.constant_initializer(0.0),
+                                        padding='SAME', name="conv1_2", kernel_initializer=weight_init)  # bx200x200x64
+            self.conv1_2 = slim.batch_norm(self.conv1_2)
+            self.conv1_2 = tf.nn.relu(self.conv1_2)
+
+            self.output1 = self.side_layer(self.conv1_2,name='output1',filters=1, upscale=int(2 ** 1),
+                                           strides=(1,1),kernel_size=[1,1],sub_pixel=use_subpixel,
+                                           kernel_init=weight_init)  # bx400x400x1
+            self.rconv1 = tf.layers.conv2d(
+                self.conv1_2,filters=128, kernel_size=[1,1], activation=None,
+                strides=(2,2), bias_initializer=tf.constant_initializer(0.0),
+                padding='SAME', name="rconv1", kernel_initializer=weight_init)  # bx100x100x128 --Skip left
+            self.rconv1 =slim.batch_norm(self.rconv1) # bx100x100x128 --Skip left
+
+            # ------------------------- Block2 ----------------------------------------
+            self.block2_xcp = self.conv1_2
+            for k in range(1):
+                self.block2_xcp = tf.layers.conv2d(
+                    self.block2_xcp, filters=128, kernel_size=[3, 3],
+                    strides=(1, 1), padding='same', name='conv_block2_{}'.format(k + 1),
+                kernel_initializer=weight_init) # bx200x200x128
+                self.block2_xcp = slim.batch_norm(self.block2_xcp)
+                self.block2_xcp = tf.nn.relu(self.block2_xcp)
+
+                self.block2_xcp = tf.layers.conv2d(
+                    self.block2_xcp, filters=128, kernel_size=[3, 3],
+                    strides=(1, 1), padding='same', name='conv2_block2_{}'.format(k + 1),
+                kernel_initializer=weight_init) # bx200x200x128
+                self.block2_xcp= slim.batch_norm(self.block2_xcp)
+
+            self.maxpool2_1=slim.max_pool2d(self.block2_xcp,kernel_size=[3,3],stride=2, padding='same',
+                                        scope='maxpool2_1') # bx100x100x128
+            self.add2_1 = tf.add(self.maxpool2_1, self.rconv1)# with skip left
+            self.output2 = self.side_layer(self.block2_xcp,filters=1,name='output2', upscale=int(2 ** 1),
+                                           strides=(1,1),kernel_size=[1,1],sub_pixel=use_subpixel,
+                                           kernel_init=weight_init) # bx400x400x1
+            self.rconv2= tf.layers.conv2d(
+                self.add2_1,filters=256, kernel_size=[1,1], activation=None,
+                kernel_initializer=weight_init, strides=(2,2), bias_initializer=tf.constant_initializer(0.0),
+                padding='SAME', name="rconv2")  # bx50x50x256 # skip left
+            self.rconv2 = slim.batch_norm(self.rconv2)  # skip left
+
+            # ------------------------- Block3 ----------------------------------------
+            self.block3_xcp = self.add2_1
+            self.addb2_4b3 = tf.layers.conv2d(
+                self.maxpool2_1,filters=256, kernel_size=[1, 1], activation=None,
+                kernel_initializer=weight_init, strides=(1, 1), bias_initializer=tf.constant_initializer(0.0),
+                padding='SAME', name="add2conv_4b3") # 100x100x256 # skip right
+            self.addb2_4b3 = slim.batch_norm(self.addb2_4b3) # skip right
+            for k in range(2):
+
+                self.block3_xcp=tf.nn.relu(self.block3_xcp)
+                self.block3_xcp = tf.layers.conv2d(
+                    self.block3_xcp, filters=256, kernel_size=[3, 3],
+                    strides=(1, 1), padding='same', name='con1v_block3_{}'.format(k + 1),
+                kernel_initializer=weight_init) # bx100x100x256
+                self.block3_xcp = slim.batch_norm(self.block3_xcp)
+                self.block3_xcp = tf.nn.relu(self.block3_xcp)
+
+                self.block3_xcp = tf.layers.conv2d(
+                    self.block3_xcp, filters=256, kernel_size=[3, 3],
+                    strides=(1,1),padding='same',name='conv2_block3_{}'.format(k+1),
+                kernel_initializer=weight_init)  # bx100x100x256
+                self.block3_xcp = slim.batch_norm(self.block3_xcp)
+                self.block3_xcp = tf.add(self.block3_xcp, self.addb2_4b3)/2 #  with  right skip
+
+            self.maxpool3_1 = slim.max_pool2d(self.block3_xcp, kernel_size=[3, 3],stride=2, padding='same',
+                                             scope='maxpool3_1')  # bx50x50x256
+            self.add3_1 = tf.add(self.maxpool3_1, self.rconv2) # with before skip left
+            self.rconv3 = tf.layers.conv2d(
+                self.add3_1, filters=512, kernel_size=[1, 1], activation=None,
+                kernel_initializer=weight_init, strides=(2, 2), bias_initializer=tf.constant_initializer(0.0),
+                padding='SAME', name="rconv3")  # bx25x25x512  # skip left
+            self.rconv3 = slim.batch_norm(self.rconv3) # skip left
+            self.output3 = self.side_layer(self.block3_xcp, filters=1,name='output3', upscale=int(2 ** 2),
+                                           strides=(1,1),kernel_size=[1,1],sub_pixel=use_subpixel,
+                                           kernel_init=weight_init)   # bx400x400x1
+
+            # ------------------------- Block4 ----------------------------------------
+            self.conv_b2b4 = tf.layers.conv2d(
+                self.maxpool2_1, filters=256, kernel_size=[1, 1], activation=None,
+                kernel_initializer=weight_init, strides=(2, 2), bias_initializer=tf.constant_initializer(0.0),
+                padding='SAME', name="conv_b2b4")  # bx50x50x256 # skip right
+            self.block4_xcp= self.add3_1
+            self.addb2b3 = tf.add(self.conv_b2b4, self.maxpool3_1)# skip right
+            self.addb3_4b4 = tf.layers.conv2d(
+                self.addb2b3, filters=512, kernel_size=[1, 1], activation=None,
+                kernel_initializer=weight_init, strides=(1, 1), bias_initializer=tf.constant_initializer(0.0),
+                padding='SAME', name="add3conv_4b4")  # bx50x50x512 # skip right
+            self.addb3_4b4 = slim.batch_norm(self.addb3_4b4)# skip right
+            for k in range(3):
+                self.block4_xcp= tf.nn.relu(self.block4_xcp)
+                self.block4_xcp = tf.layers.conv2d(
+                    self.block4_xcp, filters=512, kernel_size=[3, 3], strides=(1, 1),
+                    padding='same', name='conv1_block4_{}'.format(k + 1), kernel_initializer=weight_init)  # bx50x50x512
+                self.block4_xcp = slim.batch_norm(self.block4_xcp)
+                self.block4_xcp = tf.nn.relu(self.block4_xcp)
+
+                self.block4_xcp = tf.layers.conv2d(
+                    self.block4_xcp, filters=512, kernel_size=[3, 3], strides=(1, 1),
+                    padding='same', name='conv2_block4_{}'.format(k+1), kernel_initializer=weight_init) # bx50x50x512
+                self.block4_xcp = slim.batch_norm(self.block4_xcp)
+                self.block4_xcp = tf.add(self.block4_xcp, self.addb3_4b4)/2 #  with  right skip
+
+            self.maxpool4_1 = slim.max_pool2d(self.block4_xcp, kernel_size=[3, 3], stride=2, padding='same',
+                                             scope='maxpool3_1')  # bx25x25x728, b=batch size
+            self.add4_1 = tf.add(self.maxpool4_1, self.rconv3) # with skip left
+            self.rconv4 = tf.layers.conv2d(
+                self.add4_1, filters=512, kernel_size=[1, 1], activation=None,
+                kernel_initializer=weight_init, strides=(1, 1), bias_initializer=tf.constant_initializer(0.0),
+                padding='SAME', name="rconv4")  # bx25x25x512  # skip leff
+            self.rconv4 = slim.batch_norm(self.rconv4)   # skip left
+
+            self.output4 = self.side_layer(self.block4_xcp, filters=1,name='output4', upscale=int(2 ** 3),
+                                           strides=(1,1),kernel_size=[1,1],sub_pixel=use_subpixel,
+                                           kernel_init=weight_init)  # bx400x400x1
+
+            # ------------------------- Block5 ----------------------------------------
+            self.convb3_2ab4 = tf.layers.conv2d(
+                self.conv_b2b4, filters=512, kernel_size=[1, 1], activation=None,
+                kernel_initializer=weight_init, strides=(2, 2), bias_initializer=tf.constant_initializer(0.0),
+                            padding='SAME', name="conv_b2b5")  # bx25x25x512  # skip right
+
+            self.block5_xcp=self.add4_1
+            self.addb2b5 =  tf.add(self.convb3_2ab4,self.maxpool4_1)  # skip right
+            self.addb2b5 = tf.layers.conv2d(
+                self.addb2b5, filters=512, kernel_size=[1, 1], activation=None,
+                kernel_initializer=weight_init, strides=(1, 1), bias_initializer=tf.constant_initializer(0.0),
+                padding='SAME', name="addb2b5")  # bx25x25x512# skip right
+            self.addb2b5 = slim.batch_norm(self.addb2b5)# skip right
+            for k in range(3):
+                self.block5_xcp=tf.nn.relu(self.block5_xcp)
+                self.block5_xcp= tf.layers.conv2d(
+                    self.block5_xcp, filters=512, kernel_size=[3, 3],
+                    strides=(1, 1),padding='SAME', name="conv1_block5{}".format(k+1),
+                kernel_initializer=weight_init)  # bx25x25x512
+                self.block5_xcp = slim.batch_norm(self.block5_xcp)
+                self.block5_xcp=tf.nn.relu(self.block5_xcp)
+
+                self.block5_xcp= tf.layers.conv2d(
+                    self.block5_xcp, filters=512, kernel_size=[3, 3],
+                    strides=(1, 1),padding='SAME', name="conv2_block5{}".format(k+1),
+                kernel_initializer=weight_init)  # bx25x25x728
+                self.block5_xcp = slim.batch_norm(self.block5_xcp)
+                self.block5_xcp=tf.add(self.block5_xcp,self.addb2b5)/2 # wwith  right skip
+
+            self.add5_1 = tf.add(self.block5_xcp, self.rconv4) # with skip left
+            self.output5 = self.side_layer(self.block5_xcp, filters=1,name='output5', kernel_size=[1,1],
+                                           upscale=int(2 ** 4), sub_pixel=use_subpixel, strides=(1,1),
+                                           kernel_init=weight_init)
+
+            # ------------------------- Block6 ----------------------------------------
+            self.block6_xcp = self.add5_1
+            self.block6_xcp = tf.layers.conv2d(
+                self.block6_xcp, filters=256, kernel_size=[1, 1], activation=None,
+                kernel_initializer=weight_init, strides=(1, 1), bias_initializer=tf.constant_initializer(0.0),
+                padding='SAME', name="conv0_b6")  # bx25x25x256
+
+            self.block6_xcp = slim.batch_norm(self.block6_xcp)
+            self.addb25_2b6 = tf.layers.conv2d(
+                self.block5_xcp, filters=256, kernel_size=[1, 1], activation=None,
+                kernel_initializer=weight_init, strides=(1, 1), bias_initializer=tf.constant_initializer(0.0),
+                padding='SAME', name="add2b6")  # bx25x25x256# skip right
+            self.addb25_2b6 = slim.batch_norm(self.addb25_2b6)# skip right
+            for k in range(3):
+                self.block6_xcp = tf.nn.relu(self.block6_xcp)
+                self.block6_xcp = tf.layers.conv2d(
+                    self.block6_xcp, filters=256, kernel_size=[3, 3],
+                    strides=(1, 1), padding='SAME', name="conv1_block6{}".format(k + 1),
+                kernel_initializer=weight_init)  # bx25x25x256
+                self.block6_xcp = slim.batch_norm(self.block6_xcp)
+                self.block6_xcp = tf.nn.relu(self.block6_xcp)
+
+                self.block6_xcp = tf.layers.conv2d(
+                    self.block6_xcp, filters=256, kernel_size=[3, 3],
+                    strides=(1, 1), padding='SAME', name="conv2_block6{}".format(k + 1),
+                kernel_initializer=weight_init)  # bx25x25x256
+                self.block6_xcp = slim.batch_norm(self.block6_xcp)
+                self.block6_xcp = tf.add(self.block6_xcp, self.addb25_2b6) / 2 #  with  right skip
+
+            self.output6 = self.side_layer(self.block6_xcp, filters=1, name='output6', kernel_size=[1, 1],
+                                           upscale=int(2 ** 4), sub_pixel=use_subpixel, strides=(1, 1),
+                                           kernel_init=weight_init)
+            # ******************** End blocks *****************************************
+
+            self.side_outputs = [self.output1, self.output2, self.output3,
+                                 self.output4, self.output5,self.output6]
+
+            self.fuse = tf.layers.conv2d(tf.concat(self.side_outputs, axis=3),filters=1,
+                                        kernel_size=[1,1], name='fuse_1',strides=(1,1),padding='same',
+                                        kernel_initializer=tf.constant_initializer(1 / len(self.side_outputs)))
+            self.outputs = self.side_outputs + [self.fuse]
+
+        print_info("Build model finished: {:.4f}s".format(time.time() - start_time))
+
+    def max_pool(self, bottom, name):
+        return tf.nn.max_pool(bottom, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME', name=name)
+
+    def conv_layer(self, inputs, filters=None,kernel_size=None, depth_multiplier=None,
+                   padding='same', activation=None, name=None,
+                   kernel_initializer=None, strides=(1,1), separable_conv=False):
+
+        if separable_conv:
+            conv = tf.layers.separable_conv2d(
+                inputs, filters=filters, kernel_size=kernel_size,
+                depth_multiplier=depth_multiplier, padding=padding, name=name)
+        else:
+            conv= tf.layers.conv2d(inputs, filters=filters, kernel_size=kernel_size,
+                             strides=strides,padding=padding, kernel_initializer=kernel_initializer, name=name)
+        return conv
+
+    def side_layer(self, inputs, filters=None,kernel_size=None, strides=(1,1),
+                   name=None, upscale=None, sub_pixel=False,kernel_init=None):
+        """
+            https://github.com/s9xie/hed/blob/9e74dd710773d8d8a469ad905c76f4a7fa08f945/examples/hed/train_val.prototxt#L122
+            1x1 conv followed with Deconvoltion layer to upscale the size of input image sans color
+        """
+        def upsample_block(inputs, filters=None,kernel_size=None, strides=(1,1),
+                   name=None, upscale=None, sub_pixel=False):
+            i=1
+            scale=2
+            sub_net=inputs
+            output_filters=16
+            if sub_pixel is None:
+                # Upsampling by transpose_convolution
+                while (scale<=upscale):
+                    if scale==upscale:
+
+                        sub_net = self.conv_layer(sub_net, filters=filters, kernel_size=kernel_size,
+                                                  strides=strides,kernel_initializer=tf.truncated_normal_initializer(mean=0.0),
+                                                  name=name + '_conv_{}'.format(i))  # bx100x100x64
+                        biases = tf.Variable(tf.constant(0.0, shape=[filters], dtype=tf.float32),
+                                             name=name + '_biases_{}'.format(i))
+                        sub_net = tf.nn.bias_add(sub_net, biases)
+                        sub_net = tf.nn.relu(sub_net)
+
+                        sub_net = tf.layers.conv2d_transpose(
+                            sub_net, filters=filters, kernel_size=[(upscale), (upscale)],
+                            strides=(2, 2), padding="SAME", kernel_initializer=tf.truncated_normal_initializer(stddev=0.1),
+                            name='{}_deconv_{}_{}'.format(name, upscale, i)) # upscale/2
+                    else:
+
+                        sub_net = self.conv_layer(sub_net, filters=output_filters,
+                                                  kernel_size=kernel_size,kernel_initializer=kernel_init,
+                                                  strides=strides, name=name + '_conv_{}'.format(i))  # bx100x100x64 tf.truncated_normal_initializer(mean=0.0, stddev=0.15)
+                        biases = tf.Variable(tf.constant(0.0, shape=[output_filters], dtype=tf.float32),
+                                             name=name + '_biases_{}'.format(i))
+
+                        sub_net = tf.nn.bias_add(sub_net, biases)
+                        sub_net = tf.nn.relu(sub_net)
+                        # *
+                        sub_net = tf.layers.conv2d_transpose(
+                            sub_net, filters=output_filters, kernel_size=[(upscale), (upscale)],
+                            strides=(2, 2), padding="SAME", kernel_initializer=kernel_init,
+                            name='{}_deconv_{}_{}'.format(name, upscale, i))
+                    i += 1
+                    scale=2**i
+
+            elif sub_pixel is False:
+                # Upsampling by bilinear interpolation
+                while (scale <= upscale):
+
+                    if scale == upscale:
+                        cur_shape = sub_net.get_shape().as_list()
+                        sub_net = self.conv_layer(sub_net, filters=1,
+                                                  kernel_size=3, kernel_initializer=kernel_init,
+                                                  strides=strides, name=name + '_conv'+str(i))  # bx100x100x64 tf.truncated_normal_initializer(mean=0.0, stddev=0.15)
+                        biases = tf.Variable(tf.constant(0.0, shape=[1], dtype=tf.float32),
+                                             name=name + '_conv_b'+str(i))
+                        sub_net = tf.nn.bias_add(sub_net, biases)
+                        sub_net = tf.nn.relu(sub_net)
+                        if cur_shape[1]== self.img_height and cur_shape[2]==self.img_width:
+                            pass
+                        else:
+                            sub_net = self._upscore_layer(input=sub_net,n_outputs=1,stride=upscale,ksize=upscale,
+                                                      name=name+'_bdconv'+str(i))
+                    else:
+                        cur_shape = sub_net.get_shape().as_list()
+                        sub_net = self.conv_layer(sub_net, filters=output_filters,
+                                                  kernel_size=3, kernel_initializer=kernel_init,
+                                                  strides=strides, name=name + '_conv' + str(
+                                i))  # bx100x100x64 tf.truncated_normal_initializer(mean=0.0, stddev=0.15)
+                        biases = tf.Variable(tf.constant(0.0, shape=[output_filters], dtype=tf.float32),
+                                             name=name + '_conv_b' + str(i))
+                        sub_net = tf.nn.bias_add(sub_net, biases)
+                        sub_net = tf.nn.relu(sub_net)
+                        if cur_shape[1] == self.img_height and cur_shape[2] == self.img_width:
+                            pass
+                        else:
+                            sub_net = self._upscore_layer(input=sub_net, n_outputs=output_filters, stride=upscale, ksize=upscale,
+                                                          name=name + '_bdconv' + str(i))
+                    i += 1
+                    scale = 2 ** i
+
+            elif sub_pixel is True:
+                # Upsampling by subPixel convolution
+                while (scale <= upscale):
+                    if scale == upscale:
+                        sub_net = self.conv_layer(sub_net, filters=4,
+                                                  kernel_size=3, kernel_initializer=kernel_init,
+                                                  strides=strides, name=name + '_conv'+str(i))  # bx100x100x64 tf.truncated_normal_initializer(mean=0.0, stddev=0.15)
+                        biases = tf.Variable(tf.constant(0.0, shape=[4], dtype=tf.float32),
+                                             name=name + '_conv_b'+str(i))
+                        sub_net = tf.nn.bias_add(sub_net, biases)
+                        sub_net = tf.nn.relu(sub_net)
+                        _err_log = "SubpixelConv2d: The number of input channels == (scale x scale)" \
+                                   " x The number of output channels"
+                        r = 2
+                        if filters >= 1:
+                            if int(sub_net.get_shape()[-1]) != int(r ** 2 * filters):
+                                raise Exception(_err_log)
+                            sub_net = tf.depth_to_space(sub_net, r)
+                        else:
+                            raise Exception(' the output channel is not setted')
+                    else:
+
+                        sub_net = self.conv_layer(
+                            sub_net, filters=32, kernel_size=3, kernel_initializer=kernel_init,
+                            strides=strides, name=name + '_conv' + str(i))  # bx100x100x32
+                        biases = tf.Variable(tf.constant(0.0, shape=[32], dtype=tf.float32),
+                                             name=name + '_conv_b' + str(i))
+                        sub_net = tf.nn.bias_add(sub_net, biases)
+                        sub_net = tf.nn.relu(sub_net)
+                        _err_log = "SubpixelConv2d: The number of input channels == (scale x scale)" \
+                                   " x The number of output channels"
+                        r = 2
+                        sp_filter =8
+                        if sp_filter >= 1:
+                            if int(sub_net.get_shape()[-1]) != int(r ** 2 * sp_filter):
+                                raise Exception(_err_log)
+                            sub_net = tf.nn.depth_to_space(sub_net, r)
+                        else:
+                            raise Exception(' the output channel is not setted')
+                    i += 1
+                    scale = 2 ** i
+            else:
+                raise NotImplementedError
+            return sub_net
+        classifier = upsample_block(inputs, filters=filters, kernel_size=kernel_size, strides=strides,
+                                        name=name, upscale=upscale, sub_pixel=sub_pixel)
+        return classifier
+
+    def _upscore_layer(self, input, n_outputs, name,
+                       ksize=4, stride=2,shape=None):
+        strides = [1, stride, stride, 1]
+        in_features = input.get_shape().as_list()[3]
+
+        if shape is None:
+            # Compute shape out of Bottom
+            in_shape = tf.shape(input)
+            ot_shape = input.get_shape().as_list()
+
+            h = ((ot_shape[1] - 1) * stride) + 1
+            w = ((ot_shape[2] - 1) * stride) + 1
+            # new_shape = [in_shape[0], h, w, n_outputs]
+            new_shape = [in_shape[0], self.img_height, self.img_width, n_outputs] #output_shape=[,]
+        else:
+            new_shape = [shape[0], shape[1], shape[2], n_outputs]
+        output_shape = tf.stack(new_shape)
+
+        f_shape = [ksize, ksize, n_outputs, in_features]
+        # create
+        num_input = ksize * ksize * in_features / stride
+        stddev = (2 / num_input) ** 0.5
+        weights = self.get_deconv_filter(f_shape,name=name+'_Wb')
+        deconv = tf.nn.conv2d_transpose(input,weights, output_shape,
+                                        strides=strides, padding='SAME', name=name)
+        # _activation_summary(deconv)
+        return deconv
+
+    def get_deconv_filter(self, f_shape,name=''):
+        width = f_shape[0]
+        heigh = f_shape[0]
+        f = np.ceil(width / 2.0)
+        c = (2 * f - 1 - f % 2) / (2.0 * f)
+        bilinear = np.zeros([f_shape[0], f_shape[1]])
+        for x in range(width):
+            for y in range(heigh):
+                value = (1 - abs(x / f - c)) * (1 - abs(y / f - c))
+                bilinear[x, y] = value
+        weights = np.zeros(f_shape)
+        for i in range(f_shape[2]):
+            weights[:, :, i, i] = bilinear
+
+        init = tf.constant_initializer(value=weights,
+                                       dtype=tf.float32)
+
+        return tf.get_variable(name=name, initializer=init, shape=weights.shape)
+
+    def setup_testing(self, session):
+        """
+            Apply sigmoid non-linearity to side layer ouputs + fuse layer outputs for predictions
+        """
+        self.predictions = []
+
+        for idx, b in enumerate(self.outputs):
+            output = tf.nn.sigmoid(b, name='output_{}'.format(idx))
+            self.predictions.append(output)
+
+    def setup_training(self, session):
+        """
+            Apply sigmoid non-linearity to side layer ouputs + fuse layer outputs
+            Compute total loss := side_layer_loss + fuse_layer_loss
+            Compute predicted edge maps from fuse layer as pseudo performance metric to track
+        """
+        self.predictions = []
+        self.loss = 0
+        self.fuse_output = []
+        self.losses=[]
+
+        print_warning('Deep supervision application set to {}'.format(self.args.deep_supervision))
+        ci=np.arange(len(self.side_outputs))
+        for idx, b in enumerate(self.side_outputs):
+            output = tf.nn.sigmoid(b, name='output_{}'.format(idx))
+            if self.args.deep_supervision and idx in ci:
+                cost = sigmoid_cross_entropy_balanced(b, self.edgemaps, name='cross_entropy{}'.format(idx)) # Deep supervision
+                self.loss += (self.args.loss_weights * cost) # Deep supervision
+                self.predictions.append(output)
+            else:
+                self.predictions.append(output)
+
+        # loss for the last side
+        self.fuse_output = tf.nn.sigmoid(self.fuse, name='fuse')  # self by me
+        fuse_cost = sigmoid_cross_entropy_balanced(self.fuse, self.edgemaps, name='cross_entropy_fuse')
+
+        self.predictions.append(self.fuse_output)
+        self.loss += (self.args.loss_weights * fuse_cost) if self.args.deep_supervision else  fuse_cost# deep supervision
+
+        pred = tf.cast(tf.greater(self.fuse_output, 0.5), tf.int32, name='predictions')
+        error = tf.cast(tf.not_equal(pred, tf.cast(self.edgemaps, tf.int32)), tf.float32)
+        self.error = tf.reduce_mean(error, name='pixel_error')
+
+        tf.compat.v1.summary.scalar('Training', self.loss)
+        tf.compat.v1.summary.scalar('Validation', self.error)
+
+        self.merged_summary = tf.compat.v1.summary.merge_all()
+
+        self.train_log_dir = os.path.join(self.args.logs_dir,
+                                      os.path.join(self.args.model_name+'_'+self.args.train_dataset,'train'))
+        self.val_log_dir = os.path.join(self.args.logs_dir,
+                                    os.path.join(self.args.model_name+'_'+self.args.train_dataset, 'val'))
+
+        if not os.path.exists(self.train_log_dir):
+            os.makedirs(self.train_log_dir)
+        if not os.path.exists(self.val_log_dir):
+            os.makedirs(self.val_log_dir)
+        self.train_writer = tf.compat.v1.summary.FileWriter(self.train_log_dir, session.graph)
+        self.val_writer = tf.compat.v1.summary.FileWriter(self.val_log_dir)

legacy/requirements.txt

@@ -0,0 +1,33 @@
+absl-py==0.9.0
+cachetools==4.1.0
+certifi==2020.4.5.1
+chardet==3.0.4
+cycler==0.10.0
+gast==0.2.2
+google-auth==1.14.1
+google-auth-oauthlib==0.4.1
+google-pasta==0.1.8
+grpcio==1.26.0
+h5py==2.10.0
+idna==2.9
+Keras-Applications==1.0.8
+Keras-Preprocessing==1.1.0
+Markdown==3.1.1
+matplotlib==3.1.2
+mock==4.0.2
+numpy==1.22
+oauthlib==3.1.0
+opencv-python==4.4.0
+opt-einsum==3.2.1
+pkg-resources==0.0.0
+protobuf==3.15.0
+pyasn1==0.4.8
+pyasn1-modules==0.2.8
+pyparsing==2.4.6
+python-dateutil==2.8.1
+requests==2.23.0
+requests-oauthlib==1.3.0
+six==1.14.0
+termcolor==1.1.0
+Werkzeug==0.16.1
+wrapt==1.11.2

legacy/run_model.py

@@ -0,0 +1,134 @@
+""" DexiNed main script
+
+This code is based on DexiNed (Dense Extreme Inception Network for Edge Detection),
+Please pay attention in the function config_model() to set any parameter before training or
+testing the model.
+"""
+__author__ = "Xavier Soria Poma, CVC-UAB"
+__email__ = "[email protected] / [email protected]"
+__homepage__="www.cvc.uab.cat/people/xsoria"
+__credits__=['DexiNed']
+__copyright__   = "MIT License [see LICENSE for details]"#"Copyright 2019, CIMI"
+
+import sys
+import argparse
+import tensorflow as tf
+
+import utls.dataset_manager as dm
+from train import m_trainer
+from test import m_tester
+import platform
+
+def config_model():
+    in_linux = True if platform.system() == "Linux" else False
+    base_dir = "/opt/dataset/" if in_linux else "../../dataset/"
+    parser = argparse.ArgumentParser(description='Basic details to run HED')
+    # dataset config
+    parser.add_argument('--train_dataset', default='BIPED', choices=['BIPED','BSDS'])
+    parser.add_argument('--test_dataset', default='CLASSIC', choices=['BIPED', 'BSDS','MULTICUE','NYUD','PASCAL','CID','DCD'])
+    parser.add_argument('--dataset_dir',default=base_dir,type=str) # default:'/opt/dataset/'
+    parser.add_argument('--dataset_augmented', default=True,type=bool)
+    parser.add_argument('--train_list',default='train_rgb.lst', type=str) # BSDS train_pair.lst, SSMIHD train_rgb_pair.lst/train_rgbn_pair.lst
+    parser.add_argument('--test_list', default='test_rgb.lst',type=str) # for NYUD&BSDS:test_pair.lst, biped msi_test.lst/test_rgb.lst
+    parser.add_argument('--trained_model_dir', default='train',type=str) # 'trainV2_RN'
+    # SSMIHD_RGBN msi_valid_list.txt and msi_test_list.txt is for unified test
+    parser.add_argument('--use_nir', default=False, type=bool)
+    parser.add_argument('--use_dataset', default=False, type=bool) # test: dataset=True single image=FALSE
+    # model config
+    parser.add_argument('--model_state', default='train', choices=['train','test','None']) # always in None
+    parser.add_argument('--model_name', default='DXN',choices=['DXN','XCP','None'])
+    parser.add_argument('--use_v1', default=False,type=bool)
+    parser.add_argument('--model_purpose', default='edges',choices=['edges','restoration','None'])
+    parser.add_argument('--batch_size_train',default=8,type=int)
+    parser.add_argument('--batch_size_val',default=8, type=int)
+    parser.add_argument('--batch_size_test',default=1,type=int)
+    parser.add_argument('--checkpoint_dir', default='checkpoints',type=str)
+    parser.add_argument('--logs_dir', default='logs',type=str)
+    parser.add_argument('--learning_rate',default=1e-4, type=float) # 1e-4=0.0001
+    parser.add_argument('--lr_scheduler',default=None,choices=[None,'asce','desc']) # check here
+    parser.add_argument('--learning_rate_decay', default=0.1,type=float)
+    parser.add_argument('--weight_decay', default=0.0002, type=float)
+    parser.add_argument('--model_weights_path', default='vgg16_.npy')
+    parser.add_argument('--train_split', default=0.9, type=float) # default 0.8
+    parser.add_argument('--max_iterations', default=180000, type=int) # 100000
+    parser.add_argument('--learning_decay_interval',default=25000, type=int) # 25000
+    parser.add_argument('--loss_weights', default=1.0, type=float)
+    parser.add_argument('--save_interval', default=20000, type=int)  # 50000
+    parser.add_argument('--val_interval', default=30, type=int)
+    parser.add_argument('--use_subpixel', default=None, type=bool)  # None=upsampling with transp conv
+    parser.add_argument('--deep_supervision', default=True, type= bool)
+    parser.add_argument('--target_regression',default=True, type=bool) # true
+    parser.add_argument('--mean_pixel_values', default=[103.939,116.779,123.68, 137.86], type=float)# [103.939,116.779,123.68]
+    # for Nir pixels mean [103.939,116.779,123.68, 137.86]
+    parser.add_argument('--channel_swap', default=[2,1,0], type=int)
+    parser.add_argument('--gpu-limit',default=1.0, type= float, )
+    parser.add_argument('--use_trained_model', default=True, type=bool)  # for vvg16
+    parser.add_argument('--use_previous_trained', default=False, type=bool) # for training
+    # image configuration
+    parser.add_argument('--image_width', default=512, type=int) # 480 NYUD=560 BIPED=1280 default 400 other 448
+    parser.add_argument('--image_height', default=512, type=int) # 480 for NYUD 425 BIPED=720 default 400
+    parser.add_argument('--n_channels', default=3, type=int) # last ssmihd_xcp trained in 512
+    # test config
+    parser.add_argument('--test_snapshot', default=149999, type=int) #  BIPED: 149736 BSDS:101179
+    #DexiNedv1=149736,DexiNedv2=149999
+    parser.add_argument('--testing_threshold', default=0.0, type=float)
+    parser.add_argument('--base_dir_results',default='results/edges',type=str) # default: '/opt/results/edges'
+    # single image default=None
+    args = parser.parse_args()
+    return args
+
+def get_session(gpu_fraction):
+
+    num_threads = False
+    gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=gpu_fraction)
+
+    if num_threads:
+        return tf.compat.v1.Session(config=tf.compat.v1.ConfigProto(gpu_options=gpu_options, intra_op_parallelism_threads=num_threads))
+    else:
+        return tf.compat.v1.Session(config=tf.compat.v1.ConfigProto())
+
+
+def main(args):
+
+    if not args.dataset_augmented:
+        # Only for BIPED dataset
+        # dm.augment_data(args)
+        print("Please visit the webpage of BIPED in:")
+        print("https://xavysp.github.io/MBIPED/")
+        print("and run the code")
+        sys.exit()
+
+    if args.model_state =='train' or args.model_state=='test':
+        sess = get_session(args.gpu_limit)
+        # sess =tf.Session()
+    else:
+        print("The model state is None, so it will exit...")
+        sys.exit()
+
+    if args.model_state=='train':
+        trainer = m_trainer(args)
+        trainer.setup()
+        trainer.run(sess)
+        sess.close()
+
+    if args.model_state=='test':
+
+        if args.test_dataset=="BIPED":
+            if args.image_width >700:
+                pass
+            else:
+                print(' image size is not set in non augmented data')
+                sys.exit()
+        tester = m_tester(args)
+        tester.setup(sess)
+        tester.run(sess)
+        sess.close()
+
+    if args.model_state=="None":
+        print("Sorry the model state is {}".format(args.model_state))
+        sys.exit()
+
+if __name__=='__main__':
+
+    args = config_model()
+    main(args=args)

legacy/test.py

@@ -0,0 +1,129 @@
+
+
+import tensorflow as tf
+from PIL import Image
+
+from models.dexined import dexined
+# from models.dexinedBs import dexined
+from utls.utls import *
+from utls.dataset_manager import data_parser,get_single_image,\
+    get_testing_batch
+
+class m_tester():
+
+    def __init__(self, args):
+        self.args = args
+
+    def setup(self, session):
+
+        try:
+            if self.args.model_name=='DXN':
+                self.model = dexined(self.args)
+            else:
+                print_error("Error setting model, {}".format(self.args.model_name))
+            if self.args.trained_model_dir is None:
+                meta_model_file = os.path.join(
+                    self.args.checkpoint_dir, os.path.join(
+                        self.args.model_name + '_' + self.args.train_dataset,
+                        os.path.join('train',
+                                     '{}-{}'.format(self.args.model_name, self.args.test_snapshot))))
+            else:
+                meta_model_file = os.path.join(
+                    self.args.checkpoint_dir, os.path.join(
+                        self.args.model_name + '_' + self.args.train_dataset,
+                        os.path.join(self.args.trained_model_dir,
+                                     '{}-{}'.format(self.args.model_name, self.args.test_snapshot))))
+
+
+            saver = tf.train.Saver()
+            saver.restore(session, meta_model_file)
+            print_info('Done restoring DexiNed model from {}'.format(meta_model_file))
+
+        except Exception as err:
+
+            print_error('Error setting up DexiNed traied model, {}'.format(err))
+
+    def run(self, session):
+
+        self.model.setup_testing(session)
+        if self.args.use_dataset:
+            test_data= data_parser(self.args)
+            n_data = len(test_data[1])
+        else:
+            test_data=get_single_image(self.args)
+            n_data = len(test_data)
+        print_info('Writing PNGs at {}'.format(self.args.base_dir_results))
+
+        if self.args.batch_size_test==1 and self.args.use_dataset:
+            for i in range(n_data):
+                im, em, file_name = get_testing_batch(self.args,
+                                    [test_data[0][test_data[1][i]], test_data[1][i]], use_batch=False)
+                self.img_info = file_name
+                edgemap = session.run(self.model.predictions, feed_dict={self.model.images: [im]})
+
+                self.save_egdemaps(edgemap, single_image=True)
+                print_info('Done testing {}, {}'.format(self.img_info[0], self.img_info[1]))
+
+        # for individual images
+        elif self.args.batch_size_test==1 and not self.args.use_dataset:
+            for i in range(n_data):
+                im, file_name = get_single_image(self.args,file_path=test_data[i])
+                self.img_info  = file_name
+                edgemap = session.run(self.model.predictions, feed_dict={self.model.images: [im]})
+                self.save_egdemaps(edgemap, single_image=True)
+                print_info('Done testing {}, {}'.format(self.img_info[0], self.img_info[1]))
+
+    def save_egdemaps(self, em_maps, single_image=False):
+        """ save_edgemaps descriptios
+
+        :param em_maps:
+        :param single_image:
+        save predicted edge maps
+        """
+        result_dir = 'DexiNed_'+self.args.train_dataset+'2'+self.args.test_dataset
+        if self.args.base_dir_results is None:
+            res_dir = os.path.join('../results', result_dir)
+        else:
+            res_dir = os.path.join(self.args.base_dir_results,result_dir)
+        gt_dir = os.path.join(res_dir,'gt')
+        all_dir = os.path.join(res_dir,'pred-h5')
+        resf_dir = os.path.join(res_dir,'pred-f')
+        resa_dir = os.path.join(res_dir,'pred-a')
+
+        os.makedirs(resf_dir, exist_ok=True)
+        os.makedirs(resa_dir, exist_ok=True)
+        os.makedirs(gt_dir, exist_ok=True)
+        os.makedirs(all_dir, exist_ok=True)
+
+        if single_image:
+            em_maps = [e[0] for e in em_maps]
+            em_a = np.mean(np.array(em_maps), axis=0)
+            em_maps = em_maps + [em_a ]
+
+            em = em_maps[len(em_maps)-2]
+            em[em < self.args.testing_threshold] = 0.0
+            em_a[em_a < self.args.testing_threshold] = 0.0
+            em = 255.0 * (1.0 - em)
+            em_a = 255.0 * (1.0 - em_a)
+            em = np.tile(em, [1, 1, 3])
+            em_a = np.tile(em_a, [1, 1, 3])
+
+            em = Image.fromarray(np.uint8(em))
+            em_a = Image.fromarray(np.uint8(em_a))
+
+            tmp_name = os.path.basename(self.img_info[0])
+            tmp_name = tmp_name[:-4]
+            tmp_size = self.img_info[-1][:2]
+            tmp_size = (tmp_size[1],tmp_size[0])
+
+            em_f = em.resize(tmp_size)
+            em_a = em_a.resize(tmp_size)
+
+            em_f.save(os.path.join(resf_dir, tmp_name + '.png'))
+            em_a.save(os.path.join(resa_dir, tmp_name + '.png'))
+            em_maps =tensor_norm_01(em_maps)
+            save_variable_h5(os.path.join(all_dir, tmp_name + '.h5'), np.float16(em_maps))
+
+
+        else:
+            pass

legacy/train.py

@@ -0,0 +1,150 @@
+
+
+import tensorflow as tf
+import matplotlib.pyplot as plt
+
+from models.dexined import dexined
+# from models.dexinedBs import dexined
+from utls.utls import *
+from utls.dataset_manager import (data_parser,
+                                  get_training_batch,get_validation_batch, visualize_result)
+
+class m_trainer():
+
+    def __init__(self,args ):
+        self.init = True
+        self.args = args
+
+    def setup(self):
+        try:
+            if self.args.model_name=='DXN':
+                self.model = dexined(self.args)
+            else:
+                print_error("Error setting model, {}".format(self.args.model_name))
+
+            print_info("DL model Set")
+        except Exception as err:
+            print_error("Error setting up DL model, {}".format(err))
+            self.init=False
+
+    def run(self, sess):
+        if not self.init:
+            return
+        train_data = data_parser(self.args)
+
+        self.model.setup_training(sess)
+        if self.args.lr_scheduler is not None:
+            global_step = tf.Variable(0, trainable=False, dtype=tf.int64)
+        if self.args.lr_scheduler is None:
+            learning_rate = tf.constant(self.args.learning_rate, dtype=tf.float16)
+        else:
+            raise NotImplementedError('Learning rate scheduler type [%s] is not implemented',
+                                      self.args.lr_scheduler)
+        opt = tf.compat.v1.train.AdamOptimizer(learning_rate)
+        trainG = opt.minimize(self.model.loss)# like hed
+        saver = tf.compat.v1.train.Saver(max_to_keep=7)
+
+        sess.run(tf.compat.v1.global_variables_initializer())
+        # here to recovery previous training
+        if self.args.use_previous_trained:
+            if self.args.dataset_name.lower()!='biped': # using biped pretrained to use in other dataset
+                model_path = os.path.join(self.args.checkpoint_dir,self.args.model_name+
+                                          '_'+self.args.train_dataset,'train')
+            else:
+                model_path = os.path.join(self.args.checkpoint_dir, self.args.model_name + '_' + self.args.train_dataset)
+                model_path = os.path.join(model_path, 'train')
+            if not os.path.exists(model_path) or len(os.listdir(model_path))==0: # :
+                ini = 0
+                maxi = self.args.max_iterations+1
+                print_warning('There is not previous trained data for the current model... and')
+                print_warning('*** The training process is starting from scratch ***')
+            else:
+                # restoring using the last checkpoint
+                assert (len(os.listdir(model_path)) != 0),'There is not previous trained data for the current model...'
+                last_ckpt = tf.train.latest_checkpoint(model_path)
+                saver.restore(sess,last_ckpt)
+                ini=self.args.max_iterations
+                maxi=ini+self.args.max_iterations+1 # check
+                print_info('--> Previous model restored successfully: {}'.format(last_ckpt))
+        else:
+            print_warning('*** The training process is starting from scratch ***')
+            ini = 0
+            maxi = ini + self.args.max_iterations
+        prev_loss=1000.
+        prev_val = None
+        # directories for checkpoints
+        checkpoint_dir = os.path.join(
+            self.args.checkpoint_dir, self.args.model_name + '_' + self.args.train_dataset,
+            self.args.model_state)
+        os.makedirs(checkpoint_dir,exist_ok=True)
+
+        fig = plt.figure()
+        for idx in range(ini, maxi):
+
+            x_batch, y_batch,_ = get_training_batch(self.args, train_data)
+            run_metadata = tf.compat.v1.RunMetadata()
+
+            _, summary, loss,pred_maps= sess.run(
+                [trainG, self.model.merged_summary, self.model.loss, self.model.predictions],
+                feed_dict={self.model.images: x_batch, self.model.edgemaps: y_batch})
+            if idx%5==0:
+                self.model.train_writer.add_run_metadata(run_metadata,
+                                                         'step{:06}'.format(idx))
+                self.model.train_writer.add_summary(summary, idx)
+                print(time.ctime(), '[{}/{}]'.format(idx, maxi), ' TRAINING loss: %.5f' % loss,
+                  'prev_loss: %.5f' % prev_loss)
+
+            # saving trained parameters
+            save_inter = ini+self.args.save_interval
+            if prev_loss>loss:
+                saver.save(sess, os.path.join(checkpoint_dir, self.args.model_name), global_step=idx)
+                prev_loss = loss
+                print("Weights saved in the lowest loss",idx, " Current Loss",prev_loss)
+
+            if idx % self.args.save_interval == 0:
+                saver.save(sess, os.path.join(checkpoint_dir, self.args.model_name), global_step=idx)
+                prev_loss = loss
+                print("Weights saved in the interval", idx, " Current Loss",prev_loss)
+
+            # ********* for validation **********
+            if (idx+1) % self.args.val_interval== 0:
+                pause_show=0.01
+                imgs_list = []
+                img = x_batch[2][:,:,0:3]
+                gt_mp= y_batch[2]
+                imgs_list.append(img)
+                imgs_list.append(gt_mp)
+                for i in range(len(pred_maps)):
+                    tmp=pred_maps[i][2,...]
+                    imgs_list.append(tmp)
+                vis_imgs = visualize_result(imgs_list, self.args)
+                fig.suptitle("Iterac:" + str(idx + 1) + " Loss:" + '%.5f' % loss + " training")
+                fig.add_subplot(1,1,1)
+                plt.imshow(np.uint8(vis_imgs))
+
+                print("Evaluation in progress...")
+                plt.draw()
+                plt.pause(pause_show)
+
+                im, em, _ = get_validation_batch(self.args, train_data)
+                summary, error, pred_val = sess.run(
+                    [self.model.merged_summary, self.model.error, self.model.fuse_output],
+                    feed_dict={self.model.images: im, self.model.edgemaps: em})
+                if error<=0.08:
+                    saver.save(sess, os.path.join(checkpoint_dir, self.args.model_name), global_step=idx)
+                    prev_loss = loss
+                    print("Parameters saved in the validation stage when its error is <=0.08::", error)
+
+                self.model.val_writer.add_summary(summary, idx)
+                print_info(('[{}/{}]'.format(idx, self.args.max_iterations),'VALIDATION error: %0.5f'%error,
+                           'pError: %.5f'%prev_loss))
+                if (idx+1) % (self.args.val_interval*150)== 0:
+                    print('updating visualisation')
+                    plt.close()
+                    fig = plt.figure()
+
+        saver.save(sess, os.path.join(checkpoint_dir, self.args.model_name), global_step=idx)
+        print("Final Weights saved", idx, " Current Loss", loss)
+        self.model.train_writer.close()
+        sess.close()
+

legacy/utls/dataset_manager.py

@@ -0,0 +1,1336 @@
+"""
+
+"""
+import sys
+import cv2 as cv
+from PIL import Image
+
+from legacy.utls.utls import *
+
+
+def cv_imshow(title='default',img=None):
+    print(img.shape)
+    cv.imshow(title,img)
+    cv.waitKey(0)
+    cv.destroyAllWindows()
+
+def rotated_img_extractor(x=None, gt=None,img_width=None, img_height=None,i=None, two_data=False):
+    if two_data:
+        if img_width==img_height:
+            # for images whose sizes are the same
+
+            if i % 90 == 0:
+                adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                rot_x = cv.warpAffine(x, adjus_M, (img_height, img_width))
+                rot_gt = cv.warpAffine(gt, adjus_M, (img_height, img_width))
+            elif i % 19 == 0:
+                if i == 57:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height, img_width))
+                    rot_gt = cv.warpAffine(gt, adjus_M, (img_height, img_width))
+                    # cv.rectangle(rot_x, (100, 100), (720 - 100, 720 - 100), (0, 0, 255), (2))
+                    rot_x = rot_x[100:720 - 100, 100:720 - 100, :]
+                    rot_gt = rot_gt[100:720 - 100, 100:720 - 100]
+                    # print("just for check 19: ", i, rot_x.shape)
+                elif i == 285:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height, img_width))
+                    rot_gt = cv.warpAffine(gt, adjus_M, (img_height, img_width))
+                    rot_x = rot_x[75:720 - 75, 75:720 - 75, :]
+                    rot_gt = rot_gt[75:720 - 75, 75:720 - 75]
+                else:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height, img_width))
+                    rot_gt = cv.warpAffine(gt, adjus_M, (img_height, img_width))
+                    rot_x = rot_x[95:720 - 95, 95:720 - 95, :]
+                    rot_gt = rot_gt[95:720 - 95, 95:720 - 95]
+            elif i % 23 == 0:
+                if i == 161:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height, img_width))
+                    rot_gt = cv.warpAffine(gt, adjus_M, (img_height, img_width))
+                    rot_x = rot_x[85:720 - 85, 85:720 - 85, :]
+                    rot_gt = rot_gt[85:720 - 85, 85:720 - 85]
+                else:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height, img_width))
+                    rot_gt = cv.warpAffine(gt, adjus_M, (img_height, img_width))
+                    # cv.rectangle(rot_x, (105, 105), (720 - 105, 720 - 105), (0, 0, 255), (2))
+                    rot_x = rot_x[105:720 - 105, 105:720 - 105, :]
+                    rot_gt = rot_gt[105:720 - 105, 105:720 - 105]
+                    # print("just for check 23:", i, rot_x.shape)
+
+            return rot_x, rot_gt
+        else:
+            # # for images whose sizes are ***not*** the same *********************************
+            img_size = img_width if img_width < img_height else img_height
+            if i % 90 == 0:
+                if i==180:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height+250, img_width))
+                    rot_gt = cv.warpAffine(gt, adjus_M, (img_height+250, img_width))
+                    # a = np.copy(rot_x)
+                    rot_x = rot_x[10:img_size-90, 10:img_size+110, :]
+                    rot_gt = rot_gt[10:img_size-90, 10:img_size+110]
+                else:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height + 450, img_width))
+                    rot_gt = cv.warpAffine(gt, adjus_M, (img_height + 450, img_width))
+                    # a = np.copy(rot_x)
+                    rot_x = rot_x[100:img_size + 200, 300:img_size + 200, :]
+                    rot_gt = rot_gt[100:img_size + 200, 300:img_size + 200]
+            elif i % 19 == 0:
+                if i == 57:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height+i+5, img_width))
+                    rot_gt = cv.warpAffine(gt, adjus_M, (img_height+i+5, img_width))
+                    # a = np.copy(rot_x)
+                    # #                 x    y             x           y
+                    # cv.rectangle(a, (275, 275), (img_size+55, img_size+55), (0, 0, 255), (2))
+                    #                   y                   x
+                    rot_x = rot_x[275:img_size+55, 275:img_size+55, :]
+                    rot_gt = rot_gt[275:img_size+55, 275:img_size+55]
+                    # print("just for check 19: ", i, rot_x.shape)
+                elif i == 285:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height+i, img_width))
+                    rot_gt = cv.warpAffine(gt, adjus_M, (img_height+i, img_width))
+                    rot_x = rot_x[100:img_size-50,355:img_size+205, :]
+                    rot_gt = rot_gt[100:img_size-50,355:img_size+205]
+                    # print("just for check 19: ", i, rot_x.shape)
+                elif i==19:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height+200, img_width))
+                    rot_gt = cv.warpAffine(gt, adjus_M, (img_height+200, img_width))
+                    # a = np.copy(rot_x)
+                    # cv.rectangle(a, (150, 150), (img_size+30, img_size-70), (0, 0, 255), (2))
+                    rot_x = rot_x[150:img_size-70, 150:img_size+30, :]
+                    rot_gt = rot_gt[150:img_size-70, 150:img_size+30]
+                    # print("just for check 19: ", i, rot_x.shape)
+
+                else:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height+250, img_width))
+                    rot_gt = cv.warpAffine(gt, adjus_M, (img_height+250, img_width))
+                    # a = np.copy(rot_x)
+                    # cv.rectangle(a, (400, 115), (img_size+180, img_size-105), (0, 0, 255), (2))
+                    rot_x = rot_x[115:img_size-105, 400:img_size+180, :]
+                    rot_gt = rot_gt[115:img_size-105, 400:img_size+180]
+                    # print("just for check 19: ", i, rot_x.shape)
+
+            elif i % 23 == 0:
+                if i == 161:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height+i+200, img_width))
+                    rot_gt = cv.warpAffine(gt, adjus_M, (img_height+i, img_width))
+                    # a = rot_x
+                    # cv.rectangle(a, (95, 50), (img_size+75, img_size-170), (0, 0, 255), (2))
+                    rot_x = rot_x[50:img_size-170, 95:img_size+75, :]
+                    rot_gt = rot_gt[50:img_size-170, 95:img_size+75]
+                    # print("just for check 23: ", i, rot_x.shape)
+                elif i==207:
+
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height + 250, img_width))
+                    rot_gt = cv.warpAffine(gt, adjus_M, (img_height + 250, img_width))
+                    # a = np.copy(rot_x)
+                    # cv.rectangle(a, (200, 185), (img_size + 160, img_size - 95), (0, 0, 255), (2))
+                    rot_x = rot_x[185:img_size - 95, 200:img_size + 160, :]
+                    rot_gt = rot_gt[185:img_size - 95, 200:img_size + 160]
+
+                else:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height+250, img_width))
+                    rot_gt = cv.warpAffine(gt, adjus_M, (img_height+250, img_width))
+                    # a = np.copy(rot_x)
+                    # cv.rectangle(a, (390, 115), (img_size+170, img_size-105), (0, 0, 255), (2))
+                    rot_x = rot_x[115:img_size-105, 390:img_size+170, :]
+                    rot_gt = rot_gt[115:img_size-105, 390:img_size+170]
+            return rot_x,rot_gt
+    else:
+        # For  NIR imagel but just NIR (ONE data)
+        if img_height==img_width:
+
+            if i % 90 == 0:
+                adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                rot_x = cv.warpAffine(x, adjus_M, (img_height, img_width))
+                # print("just for check 90: ", i)
+            elif i % 19 == 0:
+                if i == 57:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height, img_width))
+                    # cv.rectangle(rot_x, (100, 100), (720 - 100, 720 - 100), (0, 0, 255), (2))
+                    rot_x = rot_x[100:720 - 100, 100:720 - 100, :]
+                    # print("just for check 19: ", i, rot_x.shape)
+                elif i == 285:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height, img_width))
+                    # cv.rectangle(rot_x, (75, 75), (720 - 75, 720 - 75), (0, 0, 255), (2))
+                    rot_x = rot_x[75:720 - 75, 75:720 - 75, :]
+                    # print("just for check 19: ", i, rot_x.shape)
+                else:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height, img_width))
+                    # cv.rectangle(rot_x,(95,95),(720-95,720-95),(0,0,255),(2) )
+                    rot_x = rot_x[95:720 - 95, 95:720 - 95, :]
+                    # print("just for check 19: ", i, rot_x.shape)
+            elif i % 23 == 0:
+                if i == 161:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height, img_width))
+                    # cv.rectangle(rot_x,(85,85),(720-85,720-85),(0,0,255),(2) )
+                    rot_x = rot_x[85:720 - 85, 85:720 - 85, :]
+                    # print("just for check 23: ", i, rot_x.shape)
+                elif i==207:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height, img_width))
+                    # cv.rectangle(rot_x, (105, 105), (720 - 105, 720 - 105), (0, 0, 255), (2))
+                    rot_x = rot_x[105:720 - 105, 105:720 - 105, :]
+                    # print("just for check 23:", i, rot_x.shape)
+                else:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height, img_width))
+                    # cv.rectangle(rot_x, (105, 105), (720 - 105, 720 - 105), (0, 0, 255), (2))
+                    rot_x = rot_x[105:720 - 105, 105:720 - 105, :]
+                    # print("just for check 23:", i, rot_x.shape)
+            else:
+                print("Error line 221 in dataset_manager")
+                sys.exit()
+        else:
+            img_size = img_width if img_width < img_height else img_height
+            if i % 90 == 0:
+                if i == 180:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height + 250, img_width))
+                    rot_x = rot_x[10:img_size - 90, 10:img_size + 110, :]
+
+                else:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height + 450, img_width))
+                    rot_x = rot_x[100:img_size + 200, 300:img_size + 200, :]
+
+            elif i % 19 == 0:
+                if i == 57:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height + i + 5, img_width))
+                    # #                 x    y             x           y
+                    # cv.rectangle(a, (275, 275), (img_size+55, img_size+55), (0, 0, 255), (2))
+                    #                   y                   x
+                    rot_x = rot_x[275:img_size + 55, 275:img_size + 55, :]
+                elif i == 285:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height + i, img_width))
+                    # a = np.copy(rot_x)
+                    # cv.rectangle(a, (355, 100), (img_size+205, img_size-50), (0, 0, 255), (2))
+                    rot_x = rot_x[100:img_size - 50, 355:img_size + 205, :]
+                elif i == 19:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height + 200, img_width))
+                    # a = np.copy(rot_x)
+                    # cv.rectangle(a, (150, 150), (img_size+30, img_size-70), (0, 0, 255), (2))
+                    rot_x = rot_x[150:img_size - 70, 150:img_size + 30, :]
+
+                else:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height + 250, img_width))
+                    # a = np.copy(rot_x)
+                    # cv.rectangle(a, (400, 115), (img_size+180, img_size-105), (0, 0, 255), (2))
+                    rot_x = rot_x[115:img_size - 105, 400:img_size + 180, :]
+
+            elif i % 23 == 0:
+                if i == 161:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height + i + 200, img_width))
+                    # a = rot_x
+                    # cv.rectangle(a, (95, 50), (img_size+75, img_size-170), (0, 0, 255), (2))
+                    rot_x = rot_x[50:img_size - 170, 95:img_size + 75, :]
+                elif i == 207:
+
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height + 250, img_width))
+                    # a = np.copy(rot_x)
+                    # cv.rectangle(a, (200, 185), (img_size + 160, img_size - 95), (0, 0, 255), (2))
+                    rot_x = rot_x[185:img_size - 95, 200:img_size + 160, :]
+
+                else:
+                    adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                    rot_x = cv.warpAffine(x, adjus_M, (img_height + 250, img_width))
+                    # a = np.copy(rot_x)
+                    # cv.rectangle(a, (390, 115), (img_size+170, img_size-105), (0, 0, 255), (2))
+                    rot_x = rot_x[115:img_size - 105, 390:img_size + 170, :]
+        return rot_x
+
+def augment_data(args):
+
+    data_for = 'train'  # choice [train validation, test]
+    imgs_splitted = True
+    imgs_rotated = True
+    imgs_flipped = True
+    imgs_gamma_corrected = True
+
+    # degrees
+    #          [19, 46, 57,  90,  114,   138, 161, 180,  207, 230,   247   270,  285,  322, 342]
+    degrees = [19, 23*2, 19*3, 90, 19*6, 23*6,23*7,180, 23*9, 23*10,19*13, 270, 19*15,23*14,19*18]
+    # if data_for=='train':
+    # **************** training data ************************#
+    if data_for=='train' and not args.use_nir:
+        base_dataset_dir = args.dataset_dir.lower() + args.train_dataset + '/edges'
+        GT_dir = os.path.join(base_dataset_dir, 'edge_maps/train/rgbr/aug')
+        X_dir = os.path.join(base_dataset_dir, 'imgs/train/rgbr/aug')
+        # this implementation is just for BIPED dataset
+        gt_list = os.listdir(os.path.join(GT_dir,'real'))  #
+        gt_list.sort()
+        x_list = os.listdir(os.path.join(X_dir,'real'))
+        x_list.sort()
+        n = len(gt_list) if len(x_list)==len(gt_list) else 0
+        if n==0:
+            print('there is some inconsistence in the size of lists')
+            sys.exit()
+
+        # making 720x720 image size (splitting) ******************
+        # ********************************************************
+        if not imgs_splitted:
+            tmp_img = cv.imread(os.path.join(
+                os.path.join(X_dir, 'real'), x_list[0]))
+            img_width = tmp_img.shape[1]
+            img_height = tmp_img.shape[0]
+            if not os.path.exists(os.path.join(GT_dir, 'p1')):
+                os.makedirs(os.path.join(GT_dir, 'p1'))
+            if not os.path.exists(os.path.join(GT_dir, 'p2')):
+                os.makedirs(os.path.join(GT_dir, 'p2'))
+            if not os.path.exists(os.path.join(X_dir, 'p1')):
+                os.makedirs(os.path.join(X_dir, 'p1'))
+            if not os.path.exists(os.path.join(X_dir, 'p2')):
+                os.makedirs(os.path.join(X_dir, 'p2'))
+
+            for i in range(n):
+                x_tmp = cv.imread(os.path.join(
+                    os.path.join(X_dir,'real'), x_list[i]))
+                gt_tmp = cv.imread(os.path.join(
+                    os.path.join(GT_dir,'real'),gt_list[i]))
+                x_tmp1 = x_tmp[:,0:img_height,:]
+                x_tmp2 = x_tmp[:,img_width-img_height:img_width,:]
+
+                gt_tmp1 = gt_tmp[:,0:img_height]
+                gt_tmp2 = gt_tmp[:,img_width-img_height:img_width]
+
+                cv.imwrite(os.path.join(X_dir,os.path.join('p1',x_list[i])), x_tmp1)
+                cv.imwrite(os.path.join(X_dir, os.path.join('p2', x_list[i])), x_tmp2)
+                cv.imwrite(os.path.join(GT_dir, os.path.join('p1', gt_list[i])), gt_tmp1)
+                cv.imwrite(os.path.join(GT_dir, os.path.join('p2', gt_list[i])), gt_tmp2)
+                print('saved image: ', x_list[i], gt_list[i])
+            print('...split done')
+
+        # *************** image rotation *******************************
+        if not imgs_rotated:
+
+            # for p1 ***********
+            folder_name='real'  # choice [p1,p2,real] which are the source of files previously prepared
+            # folder_name_x = 'real'
+            if folder_name=='p1':
+                x_aug_list = os.listdir(os.path.join(X_dir, 'p1'))
+                x_aug_list.sort()
+                gt_aug_list = os.listdir(os.path.join(GT_dir, 'p1'))
+                gt_aug_list.sort()
+            elif folder_name == 'p2':
+                x_aug_list = os.listdir(os.path.join(X_dir, 'p2'))
+                x_aug_list.sort()
+                gt_aug_list = os.listdir(os.path.join(GT_dir, 'p2'))
+                gt_aug_list.sort()
+            elif folder_name=='real':
+                x_aug_list = os.listdir(os.path.join(X_dir, 'real'))
+                x_aug_list.sort()
+                gt_aug_list = os.listdir(os.path.join(GT_dir, 'real'))
+                gt_aug_list.sort()
+            else:
+                print("error reading folder name")
+                sys.exit()
+            if n==len(x_aug_list) and n== len(gt_aug_list):
+                pass
+            else:
+                print("Error reading data. The is an inconsistency in the data ")
+                sys.exit()
+
+            tmp_img = cv.imread(os.path.join(X_dir,
+                                             os.path.join(folder_name,x_aug_list[1])))
+            img_width = tmp_img.shape[1]
+            img_height = tmp_img.shape[0]
+
+            for i in (degrees):
+                if folder_name=='p1':
+                    current_X_dir =  X_dir+'/p1_rot_'+str(i)
+                    current_GT_dir = GT_dir+'/p1_rot_'+str(i)
+                elif folder_name=='p2':
+
+                    current_X_dir = X_dir + '/p2_rot_' + str(i)
+                    current_GT_dir = GT_dir + '/p2_rot_' + str(i)
+                elif folder_name=='real':
+                    current_X_dir = X_dir + '/real_rot_' + str(i)
+                    current_GT_dir = GT_dir + '/real_rot_' + str(i)
+                else:
+                    print('error')
+                    sys.exit()
+                if not os.path.exists(current_X_dir):
+                    os.makedirs(current_X_dir)
+                if not os.path.exists(current_GT_dir):
+                    os.makedirs(current_GT_dir)
+
+                for j in range(n):
+                    # i = degrees[j]
+                    if folder_name=='real':
+                        tmp_x = cv.imread(os.path.join(X_dir,
+                                                       os.path.join(folder_name, x_aug_list[j])))
+                        tmp_gt = cv.imread(os.path.join(GT_dir,
+                                                        os.path.join(folder_name, gt_aug_list[j])))
+                        rot_x,rot_gt=rotated_img_extractor(tmp_x, tmp_gt,img_width, img_height,i, True)
+                        # [19, 46, 90, 114, 138, 161, 180, 207, 230, 247, 270, 285, 322, 342]
+
+                    else:
+                        tmp_x = cv.imread(os.path.join(X_dir,
+                                               os.path.join(folder_name,x_aug_list[j])))
+                        tmp_gt = cv.imread(os.path.join(GT_dir,
+                                                        os.path.join(folder_name,gt_aug_list[j])))
+                        rot_x, rot_gt = rotated_img_extractor(tmp_x, tmp_gt, img_width, img_height, i, True)
+
+                    cv.imwrite(os.path.join(current_GT_dir,gt_aug_list[j]),rot_gt)
+                    cv.imwrite(os.path.join(current_X_dir, x_aug_list[j]),rot_x)
+                    tmp_imgs = np.concatenate((rot_x,rot_gt), axis=1)
+                    cv.imshow("rotated", tmp_imgs)
+                    cv.waitKey(400) # 1000= 1 seg
+                print("rotation with {} degrees fullfiled ".format(i))
+            cv.destroyAllWindows()
+            print("... rotation done in ", folder_name)
+
+        # **************** flipping horizontally ***********
+        if not imgs_flipped:
+            type_aug= '_flip'
+            dir_list = os.listdir(X_dir)
+            dir_list.sort()
+
+            for i in (dir_list):
+                X_list = os.listdir(os.path.join(X_dir,i))
+                X_list.sort()
+                GT_list = os.listdir(os.path.join(GT_dir,i))
+                GT_list.sort()
+                save_dir_x = X_dir+'/'+str(i)+type_aug
+                save_dir_gt = GT_dir+'/'+str(i)+type_aug
+
+                if not os.path.exists(save_dir_x):
+                    os.makedirs(save_dir_x)
+                if not os.path.exists(save_dir_gt):
+                    os.makedirs(save_dir_gt)
+
+                print("Working on the dir: ", os.path.join(X_dir,i),os.path.join(GT_dir,i) )
+                for j in range(n):
+                    x_tmp = cv.imread(os.path.join(X_dir,os.path.join(i,X_list[j])))
+                    gt_tmp = cv.imread(os.path.join(GT_dir, os.path.join(i, GT_list[j])))
+                    flip_x = np.fliplr(x_tmp)
+                    flip_gt = np.fliplr(gt_tmp)
+
+                    tmp_imgs = np.concatenate((flip_x, flip_gt), axis=1)
+                    cv.imshow("rotated", tmp_imgs)
+                    cv.waitKey(350)
+                    cv.imwrite(os.path.join(save_dir_gt,GT_list[j]),flip_gt)
+                    cv.imwrite(os.path.join(save_dir_x, X_list[j]),flip_x)
+
+                print("End flipping file in {}".format(os.path.join(X_dir,i)))
+            cv.destroyAllWindows()
+            print("... Flipping  data augmentation finished")
+
+        # ***********Data augmentation based on gamma correction **********
+        if not imgs_gamma_corrected:
+            gamma30 = '_ga30'
+            gamma60 = '_ga60'
+            dir_list = os.listdir(X_dir)
+            dir_list.sort()
+            for i in (dir_list):
+                X_list = os.listdir(os.path.join(X_dir,i))
+                X_list.sort()
+                GT_list = os.listdir(os.path.join(GT_dir,i))
+                GT_list.sort()
+                save_dir_x30 = X_dir+'/'+str(i)+gamma30
+                save_dir_gt30 = GT_dir+'/'+str(i)+gamma30
+                save_dir_x60 = X_dir + '/' + str(i)+ gamma60
+                save_dir_gt60 = GT_dir + '/'  + str(i)+ gamma60
+
+                if not os.path.exists(save_dir_x30):
+                    os.makedirs(save_dir_x30)
+                if not os.path.exists(save_dir_gt30):
+                    os.makedirs(save_dir_gt30)
+                if not os.path.exists(save_dir_x60):
+                    os.makedirs(save_dir_x60)
+                if not os.path.exists(save_dir_gt60):
+                    os.makedirs(save_dir_gt60)
+                print("Working on the dir: ", os.path.join(X_dir, i), os.path.join(GT_dir, i))
+                for j in range(n):
+                    x_tmp = cv.imread(os.path.join(X_dir, os.path.join(i, X_list[j])))
+                    gt_tmp = cv.imread(os.path.join(GT_dir, os.path.join(i, GT_list[j])))
+                    x_tmp = normalization_data_01(x_tmp)
+                    x_tmp = gamma_correction(x_tmp,0.4040,False)
+
+                    gam30_x = gamma_correction(x_tmp,0.3030,True)
+                    gam60_x = gamma_correction(x_tmp,0.6060, True)
+                    gam30_x = np.uint8(normalization_data_0255(gam30_x))
+                    gam60_x = np.uint8(normalization_data_0255(gam60_x))
+
+                    tmp_imgs1 = np.concatenate((gam30_x, gt_tmp), axis=1)
+                    tmp_imgs2 = np.concatenate((gam60_x, gt_tmp), axis=1)
+                    tmp_imgs = np.concatenate((tmp_imgs2,tmp_imgs1),axis=0)
+
+                    cv.imshow("gamma ", tmp_imgs)
+                    cv.waitKey(350)
+                    cv.imwrite(os.path.join(save_dir_gt30, GT_list[j]), gt_tmp)
+                    cv.imwrite(os.path.join(save_dir_x30, X_list[j]), gam30_x)
+                    cv.imwrite(os.path.join(save_dir_gt60, GT_list[j]), gt_tmp)
+                    cv.imwrite(os.path.join(save_dir_x60, X_list[j]), gam60_x)
+
+                print("End gamma correction, file in {}".format(os.path.join(X_dir, i)))
+            cv.destroyAllWindows()
+            print("... gamma correction  data augmentation finished")
+
+    # ************** for validation ********************************
+    elif data_for=='validation' and not args.use_nir:
+
+        train_GT_dir = args.dataset_dir + args.train_dataset + '/valid'
+        train_X_dir = args.dataset_dir + args.train_dataset + '/valid'
+
+        gt_list = os.listdir(os.path.join(train_GT_dir, 'GT_un'))  #
+        gt_list.sort()
+        x_list = os.listdir(os.path.join(train_X_dir, 'X_un'))
+        x_list.sort()
+        n = len(gt_list) if len(x_list) == len(gt_list) else 0
+        if n == 0:
+            print('there is some inconsistence in the size of lists')
+            sys.exit()
+
+        # making 720x720 image size (splitting) ******************
+        if not imgs_splitted:
+            tmp_img = cv.imread(os.path.join(
+                os.path.join(train_X_dir, 'X_un'), x_list[0]))
+            img_width = tmp_img.shape[1]
+            img_height = tmp_img.shape[0]
+            for i in range(n):
+                x_tmp = cv.imread(os.path.join(
+                    os.path.join(train_X_dir, 'X_un'), x_list[i]))
+                gt_tmp = cv.imread(os.path.join(
+                    os.path.join(train_GT_dir, 'GT_un'), gt_list[i]))
+                x_tmp1 = x_tmp[:, 0:img_height, :]
+                x_tmp2 = x_tmp[:, img_width - img_height:img_width, :]
+
+                gt_tmp1 = gt_tmp[:, 0:img_height]
+                gt_tmp2 = gt_tmp[:, img_width - img_height:img_width]
+
+                cv.imwrite(os.path.join(train_X_dir, os.path.join('X/p1', x_list[i])), x_tmp1)
+                cv.imwrite(os.path.join(train_X_dir, os.path.join('X/p2', x_list[i])), x_tmp2)
+                cv.imwrite(os.path.join(train_GT_dir, os.path.join('GT/p1', gt_list[i])), gt_tmp1)
+                cv.imwrite(os.path.join(train_GT_dir, os.path.join('GT/p2', gt_list[i])), gt_tmp2)
+                print('saved image: ', x_list[i], gt_list[i])
+
+                tmp_imgs = np.concatenate((x_tmp,gt_tmp), axis=1)
+                cv.imshow("rotated", tmp_imgs)
+                cv.waitKey(500)
+            cv.destroyAllWindows()
+            print('...split for validation finished')
+
+        # image rotation
+        if not imgs_rotated:
+            folder_name = 'p2'  # choice [p1,p2,GT_u] which are the source of files previously prepared
+            X_dir = args.dataset_dir + args.train_dataset + '/valid'
+            GT_dir = args.dataset_dir + args.train_dataset + '/valid'
+
+            folder_name_x = 'X_un'
+            if folder_name == 'p1':
+                x_aug_list = os.listdir(os.path.join(X_dir, 'X/p1'))
+                x_aug_list.sort()
+                gt_aug_list = os.listdir(os.path.join(GT_dir, 'GT/p1'))
+                gt_aug_list.sort()
+            elif folder_name == 'p2':
+                x_aug_list = os.listdir(os.path.join(X_dir, 'X/p2'))
+                x_aug_list.sort()
+                gt_aug_list = os.listdir(os.path.join(GT_dir, 'GT/p2'))
+                gt_aug_list.sort()
+            elif folder_name == 'GT_u':
+                x_aug_list = os.listdir(os.path.join(X_dir, 'X_un'))
+                x_aug_list.sort()
+                gt_aug_list = os.listdir(os.path.join(GT_dir, 'GT_un'))
+                gt_aug_list.sort()
+            else:
+                print("error reading folder name")
+                sys.exit()
+            # image size
+            tmp_img = cv.imread(os.path.join(X_dir,
+                                             os.path.join('X/' + folder_name, x_aug_list[1])))
+            img_width = tmp_img.shape[1]
+            img_height = tmp_img.shape[0]
+
+            for i in (degrees):
+                if folder_name == 'p1':
+                    current_X_dir = X_dir + '/X/p1_rot_' + str(i)
+                    current_GT_dir = GT_dir + '/GT/p1_rot_' + str(i)
+                elif folder_name == 'p2':
+
+                    current_X_dir = X_dir + '/X/p2_rot_' + str(i)
+                    current_GT_dir = GT_dir + '/GT/p2_rot_' + str(i)
+                elif folder_name == 'GT_u':
+                    current_X_dir = X_dir + 'X/un_rot_' + str(i)
+                    current_GT_dir = GT_dir + 'GT/un_rot_' + str(i)
+                else:
+                    print('error')
+                    sys.exit()
+                if not os.path.exists(current_X_dir):
+                    os.makedirs(current_X_dir)
+                if not os.path.exists(current_GT_dir):
+                    os.makedirs(current_GT_dir)
+
+                for j in range(n):
+                    if folder_name == 'GT_un':
+                        tmp_x = cv.imread(os.path.join(X_dir,
+                                                       os.path.join(folder_name_x, x_aug_list[j])))
+                        tmp_gt = cv.imread(os.path.join(GT_dir,
+                                                        os.path.join(folder_name, gt_aug_list[j])))
+                    else:
+                        tmp_x = cv.imread(os.path.join(X_dir,
+                                                       os.path.join('X/' + folder_name, x_aug_list[j])))
+                        tmp_gt = cv.imread(os.path.join(GT_dir,
+                                                        os.path.join('GT/' + folder_name, gt_aug_list[j])))
+                    if i % 90 == 0:
+                        adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                        rot_x = cv.warpAffine(tmp_x, adjus_M, (img_height, img_width))
+                        rot_gt = cv.warpAffine(tmp_gt, adjus_M, (img_height, img_width))
+                    elif i % 19 == 0:
+                        if i == 57:
+                            adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                            rot_x = cv.warpAffine(tmp_x, adjus_M, (img_height, img_width))
+                            rot_gt = cv.warpAffine(tmp_gt, adjus_M, (img_height, img_width))
+                            # cv.rectangle(rot_x, (100, 100), (720 - 100, 720 - 100), (0, 0, 255), (2))
+                            rot_x = rot_x[100:720 - 100, 100:720 - 100, :]
+                            rot_gt = rot_gt[100:720 - 100, 100:720 - 100]
+                            # print("just for check 19: ", i, rot_x.shape)
+                        elif i == 285:
+                            adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                            rot_x = cv.warpAffine(tmp_x, adjus_M, (img_height, img_width))
+                            rot_gt = cv.warpAffine(tmp_gt, adjus_M, (img_height, img_width))
+                            # cv.rectangle(rot_x, (75, 75), (720 - 75, 720 - 75), (0, 0, 255), (2))
+                            rot_x = rot_x[75:720 - 75, 75:720 - 75, :]
+                            rot_gt = rot_gt[75:720 - 75, 75:720 - 75]
+                        else:
+                            adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                            rot_x = cv.warpAffine(tmp_x, adjus_M, (img_height, img_width))
+                            rot_gt = cv.warpAffine(tmp_gt, adjus_M, (img_height, img_width))
+                            rot_x = rot_x[95:720 - 95, 95:720 - 95, :]
+                            rot_gt = rot_gt[95:720 - 95, 95:720 - 95]
+                    elif i % 23 == 0:
+                        if i == 161:
+                            adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                            rot_x = cv.warpAffine(tmp_x, adjus_M, (img_height, img_width))
+                            rot_gt = cv.warpAffine(tmp_gt, adjus_M, (img_height, img_width))
+                            # cv.rectangle(rot_x,(85,85),(720-85,720-85),(0,0,255),(2) )
+                            rot_x = rot_x[85:720 - 85, 85:720 - 85, :]
+                            rot_gt = rot_gt[85:720 - 85, 85:720 - 85]
+                            # print("just for check 23: ", i, rot_x.shape)
+                        else:
+                            adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                            rot_x = cv.warpAffine(tmp_x, adjus_M, (img_height, img_width))
+                            rot_gt = cv.warpAffine(tmp_gt, adjus_M, (img_height, img_width))
+                            rot_x = rot_x[105:720 - 105, 105:720 - 105, :]
+                            rot_gt = rot_gt[105:720 - 105, 105:720 - 105]
+                    else:
+                        print('Error using degrees for rotation')
+                        sys.exit()
+                    cv.imwrite(os.path.join(current_GT_dir, gt_aug_list[j]), rot_gt)
+                    cv.imwrite(os.path.join(current_X_dir, x_aug_list[j]), rot_x)
+                    tmp_imgs = np.concatenate((rot_x, rot_gt), axis=1)
+                    cv.imshow("rotated", tmp_imgs)
+                    cv.waitKey(1000)
+                print("rotation with {} degrees fullfiled ".format(i))
+            cv.destroyAllWindows()
+
+            print("... data rotation for validation finished ", folder_name)
+        # flipping horizontally
+        if not imgs_flipped:
+            type_aug = 'flip_'
+            X_dir = args.dataset_dir + args.train_dataset + '/valid/X'
+            GT_dir = args.dataset_dir + args.train_dataset + '/valid/GT'
+            dir_list = os.listdir(X_dir)
+            dir_list.sort()
+
+            for i in (dir_list):
+                X_list = os.listdir(os.path.join(X_dir, i))
+                X_list.sort()
+                GT_list = os.listdir(os.path.join(GT_dir, i))
+                GT_list.sort()
+                save_dir_x = X_dir + '/' + type_aug + str(i)
+                save_dir_gt = GT_dir + '/' + type_aug + str(i)
+
+                if not os.path.exists(save_dir_x):
+                    os.makedirs(save_dir_x)
+                if not os.path.exists(save_dir_gt):
+                    os.makedirs(save_dir_gt)
+
+                print("Working on the dir: ", os.path.join(X_dir, i), os.path.join(GT_dir, i))
+                for j in range(n):
+                    x_tmp = cv.imread(os.path.join(X_dir, os.path.join(i, X_list[j])))
+                    gt_tmp = cv.imread(os.path.join(GT_dir, os.path.join(i, GT_list[j])))
+                    flip_x = np.fliplr(x_tmp)
+                    flip_gt = np.fliplr(gt_tmp)
+
+                    tmp_imgs = np.concatenate((flip_x, flip_gt), axis=1)
+                    cv.imshow("rotated", tmp_imgs)
+                    cv.waitKey(1000)
+                    cv.imwrite(os.path.join(save_dir_gt, GT_list[j]), flip_gt)
+                    cv.imwrite(os.path.join(save_dir_x, X_list[j]), flip_x)
+
+                print("End flipping file in {}".format(os.path.join(X_dir, i)))
+
+            cv.destroyAllWindows()
+
+            print("... Flipping   validation stage data augmentation finished")
+
+    # ========================================================
+    # =====================Just for NIR ======================
+    # ========================================================
+    elif args.use_nir:
+
+        if data_for == 'train':
+            base_dataset_dir = args.dataset_dir + args.train_dataset + '/edges'
+            X_dir = os.path.join(base_dataset_dir, 'imgs/train/nir/aug')
+
+            x_list = os.listdir(os.path.join(X_dir, 'real'))
+            x_list.sort()
+            n = len(x_list)
+            if n == 0:
+                print('there is some inconsistence in the size of lists')
+                sys.exit()
+
+            # making 720x720 image size (splitting) ******************
+            if not imgs_splitted:
+                tmp_img = cv.imread(os.path.join(
+                    os.path.join(X_dir, 'real'), x_list[0]))
+                img_width = tmp_img.shape[1]
+                img_height = tmp_img.shape[0]
+
+                for i in range(n):
+                    x_tmp = cv.imread(os.path.join(
+                        os.path.join(X_dir, 'real'), x_list[i]))
+                    x_tmp1 = x_tmp[:, 0:img_height, :]
+                    x_tmp2 = x_tmp[:, img_width - img_height:img_width, :]
+
+                    cv.imwrite(os.path.join(X_dir, os.path.join('p1', x_list[i])), x_tmp1)
+                    cv.imwrite(os.path.join(X_dir, os.path.join('p2', x_list[i])), x_tmp2)
+                    print('saved image: ', x_list[i])
+                print('...split done')
+
+            #  ***************** image rotation ******************
+            if not imgs_rotated:
+
+                # for p1 ***********
+                folder_name = 'real'  # choice [p1,p2,real] which are the source of files previously prepared
+                # firstly X_un is not used
+                folder_name_x = 'real'
+                if folder_name == 'p1':
+                    x_aug_list = os.listdir(os.path.join(X_dir, 'p1'))
+                    x_aug_list.sort()
+                elif folder_name == 'p2':
+                    x_aug_list = os.listdir(os.path.join(X_dir, 'p2'))
+                    x_aug_list.sort()
+                elif folder_name == 'real':
+                    x_aug_list = os.listdir(os.path.join(X_dir, 'real'))
+                    x_aug_list.sort()
+                else:
+                    print("error reading folder name")
+                    sys.exit()
+
+                # image size
+                tmp_img = cv.imread(os.path.join(
+                    X_dir, os.path.join(folder_name, x_aug_list[1])))
+                img_width = tmp_img.shape[1]
+                img_height = tmp_img.shape[0]
+
+                for i in (degrees):
+                    if folder_name == 'p1':
+                        current_X_dir = X_dir + '/p1_rot_' + str(i)
+                    elif folder_name == 'p2':
+                        current_X_dir = X_dir + '/p2_rot_' + str(i)
+                    elif folder_name == 'real':
+                        current_X_dir = X_dir + '/real_rot_' + str(i)
+                    else:
+                        print('error')
+                        sys.exit()
+                    if not os.path.exists(current_X_dir):
+                        os.makedirs(current_X_dir)
+
+                    for j in range(n):
+                        # i = degrees[j]
+                        if folder_name == 'real':
+                            tmp_x = cv.imread(os.path.join(X_dir,
+                                                           os.path.join(folder_name_x, x_aug_list[j])))
+                            rot_x = rotated_img_extractor(x=tmp_x, gt=None, img_width=img_width,
+                                                          img_height=img_height, i=i, two_data=False)
+                        else:
+                            tmp_x = cv.imread(os.path.join(X_dir,
+                                                           os.path.join(folder_name, x_aug_list[j])))
+                            rot_x = rotated_img_extractor(x=tmp_x, gt=None, img_width=img_width,
+                                                          img_height=img_height, i=i, two_data=False)
+
+                        cv.imwrite(os.path.join(current_X_dir, x_aug_list[j]), rot_x)
+                        cv.imshow("rotated", rot_x)
+                        cv.waitKey(450)
+                    print("rotation with {} degrees fullfiled ".format(i))
+                cv.destroyAllWindows()
+
+                print("... rotation done in ", folder_name)
+
+            # flipping horizontally
+            if not imgs_flipped:
+                type_aug = '_flip'
+                dir_list = os.listdir(X_dir)
+                dir_list.sort()
+
+                for i in (dir_list):
+                    X_list = os.listdir(os.path.join(X_dir, i))
+                    X_list.sort()
+                    save_dir_x = X_dir + '/' + str(i)+ type_aug
+
+                    if not os.path.exists(save_dir_x):
+                        os.makedirs(save_dir_x)
+
+                    print("Working on the dir: ", os.path.join(X_dir, i), i)
+                    for j in range(n):
+                        x_tmp = cv.imread(os.path.join(X_dir, os.path.join(i, X_list[j])))
+                        flip_x = np.fliplr(x_tmp)
+
+                        cv.imshow("Flipping", x_tmp)
+                        cv.waitKey(450)
+                        cv.imwrite(os.path.join(save_dir_x, X_list[j]), flip_x)
+
+                    print("End flipping file in {}".format(os.path.join(X_dir, i)))
+
+                cv.destroyAllWindows()
+
+                print("... Flipping  data augmentation finished")
+
+            if not imgs_gamma_corrected:
+                gamma30 = '_ga30'
+                gamma60 = '_ga60'
+                dir_list = os.listdir(X_dir)
+                dir_list.sort()
+                for i in (dir_list):
+                    X_list = os.listdir(os.path.join(X_dir, i))
+                    X_list.sort()
+                    save_dir_x30 = X_dir + '/' + str(i) + gamma30
+                    save_dir_x60 = X_dir + '/' + str(i) + gamma60
+
+                    if not os.path.exists(save_dir_x30):
+                        os.makedirs(save_dir_x30)
+                    if not os.path.exists(save_dir_x60):
+                        os.makedirs(save_dir_x60)
+
+                    print("Working on the dir: ", os.path.join(X_dir, i))
+                    for j in range(n):
+                        x_tmp = cv.imread(os.path.join(X_dir, os.path.join(i, X_list[j])))
+                        x_tmp = normalization_data_01(x_tmp)
+                        x_tmp = gamma_correction(x_tmp, 0.4040, False)
+
+                        gam30_x = gamma_correction(x_tmp, 0.3030, True)
+                        gam60_x = gamma_correction(x_tmp, 0.6060, True)
+                        gam30_x = np.uint8(normalization_data_0255(gam30_x))
+                        gam60_x = np.uint8(normalization_data_0255(gam60_x))
+
+                        tmp_imgs = np.concatenate((gam30_x, gam60_x), axis=1)
+
+                        cv.imshow("gamma ", tmp_imgs)
+                        cv.waitKey(450)
+                        cv.imwrite(os.path.join(save_dir_x30, X_list[j]), gam30_x)
+                        cv.imwrite(os.path.join(save_dir_x60, X_list[j]), gam60_x)
+
+                    print("End gamma correction, file in {}".format(os.path.join(X_dir, i)))
+                cv.destroyAllWindows()
+                print("... gamma correction  data augmentation finished")
+
+
+        elif data_for == 'validation':
+
+            train_X_dir = args.dataset_dir + args.train_dataset + '/nir_valid'
+
+            x_list = os.listdir(os.path.join(train_X_dir, 'X_un'))
+            x_list.sort()
+            n = len(x_list)
+            if n == 0:
+                print('there is some inconsistence in the size of lists')
+                sys.exit()
+                # making 720x720 image size (splitting) ******************
+            if not imgs_splitted:
+                tmp_img = cv.imread(os.path.join(
+                    os.path.join(train_X_dir, 'X_un'), x_list[0]))
+                img_width = tmp_img.shape[1]
+                img_height = tmp_img.shape[0]
+
+                for i in range(n):
+                    x_tmp = cv.imread(os.path.join(
+                        os.path.join(train_X_dir, 'X_un'), x_list[i]))
+                    x_tmp1 = x_tmp[:, 0:img_height, :]
+                    x_tmp2 = x_tmp[:, img_width - img_height:img_width, :]
+
+                    cv.imwrite(os.path.join(train_X_dir, os.path.join('X/p1', x_list[i])), x_tmp1)
+                    cv.imwrite(os.path.join(train_X_dir, os.path.join('X/p2', x_list[i])), x_tmp2)
+                    print('saved image: ', x_list[i])
+                print('... validation split done')
+
+            # image rotation
+            if not imgs_rotated:
+                X_dir = args.dataset_dir + args.train_dataset + '/nir_valid'
+
+                # for p1 ***********
+                folder_name = 'p2'  # choice [p1,p2,X_un] which are the source of files previously prepared
+                # firstly X_un is not used
+                folder_name_x = 'X_un'
+                if folder_name == 'p1':
+                    x_aug_list = os.listdir(os.path.join(X_dir, 'X/p1'))
+                    x_aug_list.sort()
+                elif folder_name == 'p2':
+                    x_aug_list = os.listdir(os.path.join(X_dir, 'X/p2'))
+                    x_aug_list.sort()
+                elif folder_name == 'X_un':
+                    x_aug_list = os.listdir(os.path.join(X_dir, 'X_un'))
+                    x_aug_list.sort()
+                else:
+                    print("error reading folder name")
+                    sys.exit()
+
+                # image size
+                tmp_img = cv.imread(os.path.join(X_dir,
+                                                 os.path.join('X/' + folder_name, x_aug_list[1])))
+                img_width = tmp_img.shape[1]
+                img_height = tmp_img.shape[0]
+
+                for i in (degrees):
+                    if folder_name == 'p1':
+                        current_X_dir = X_dir + '/X/p1_rot_' + str(i)
+                    elif folder_name == 'p2':
+                        current_X_dir = X_dir + '/X/p2_rot_' + str(i)
+                    elif folder_name == 'X_un':
+                        current_X_dir = X_dir + 'X/un_rot_' + str(i)
+                    else:
+                        print('error')
+                        sys.exit()
+                    if not os.path.exists(current_X_dir):
+                        os.makedirs(current_X_dir)
+
+                    for j in range(n):
+                        # i = degrees[j]
+                        if folder_name == 'X_un':
+                            tmp_x = cv.imread(
+                                os.path.join(X_dir, os.path.join(folder_name_x, x_aug_list[j])))
+                        else:
+                            tmp_x = cv.imread(
+                                os.path.join(X_dir, os.path.join('X/' + folder_name, x_aug_list[j])))
+
+                        if i % 90 == 0:
+                            adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                            rot_x = cv.warpAffine(tmp_x, adjus_M, (img_height, img_width))
+
+                        elif i % 19 == 0:
+                            if i == 57:
+                                adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                                rot_x = cv.warpAffine(tmp_x, adjus_M, (img_height, img_width))
+                                # cv.rectangle(rot_x, (100, 100), (720 - 100, 720 - 100), (0, 0, 255), (2))
+                                rot_x = rot_x[100:720 - 100, 100:720 - 100, :]
+                                # print("just for check 19: ", i, rot_x.shape)
+                            elif i == 285:
+                                adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                                rot_x = cv.warpAffine(tmp_x, adjus_M, (img_height, img_width))
+                                # cv.rectangle(rot_x, (75, 75), (720 - 75, 720 - 75), (0, 0, 255), (2))
+                                rot_x = rot_x[75:720 - 75, 75:720 - 75, :]
+                                # print("just for check 19: ", i, rot_x.shape)
+                            else:
+                                adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                                rot_x = cv.warpAffine(tmp_x, adjus_M, (img_height, img_width))
+                                # cv.rectangle(rot_x,(95,95),(720-95,720-95),(0,0,255),(2) )
+                                rot_x = rot_x[95:720 - 95, 95:720 - 95, :]
+                                # print("just for check 19: ", i, rot_x.shape)
+
+                        elif i % 23 == 0:
+                            if i == 161:
+                                adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                                rot_x = cv.warpAffine(tmp_x, adjus_M, (img_height, img_width))
+                                # cv.rectangle(rot_x,(85,85),(720-85,720-85),(0,0,255),(2) )
+                                rot_x = rot_x[85:720 - 85, 85:720 - 85, :]
+                                # print("just for check 23: ", i, rot_x.shape)
+                            else:
+                                adjus_M = cv.getRotationMatrix2D((img_width / 2, img_height / 2), i, 1)
+                                rot_x = cv.warpAffine(tmp_x, adjus_M, (img_height, img_width))
+                                # cv.rectangle(rot_x, (105, 105), (720 - 105, 720 - 105), (0, 0, 255), (2))
+                                rot_x = rot_x[105:720 - 105, 105:720 - 105, :]
+                                # print("just for check 23:", i, rot_x.shape)
+                        else:
+                            print('Error using degrees for rotation')
+                            sys.exit()
+                        cv.imwrite(os.path.join(current_X_dir, x_aug_list[j]), rot_x)
+                        cv.imshow("rotated", rot_x)
+                        cv.waitKey(350)
+                    print("rotation with {} degrees fullfiled ".format(i))
+                cv.destroyAllWindows()
+
+                print("... rotation done in ", folder_name)
+
+            # flipping horizontally
+            if not imgs_flipped:
+                type_aug = 'flip_'
+                X_dir = args.dataset_dir + args.train_dataset + '/nir_valid/X'
+                dir_list = os.listdir(X_dir)
+                dir_list.sort()
+
+                for i in (dir_list):
+                    X_list = os.listdir(os.path.join(X_dir, i))
+                    X_list.sort()
+                    save_dir_x = X_dir + '/' + type_aug + str(i)
+
+                    if not os.path.exists(save_dir_x):
+                        os.makedirs(save_dir_x)
+
+                    print("Working on the dir: ", os.path.join(X_dir, i), i)
+                    for j in range(n):
+                        x_tmp = cv.imread(os.path.join(X_dir, os.path.join(i, X_list[j])))
+                        flip_x = np.fliplr(x_tmp)
+
+                        cv.imshow("rotated", x_tmp)
+                        cv.waitKey(350)
+                        cv.imwrite(os.path.join(save_dir_x, X_list[j]), flip_x)
+
+                    print("End flipping file in {}".format(os.path.join(X_dir, i)))
+
+                cv.destroyAllWindows()
+
+                print("... Flipping  validation data augmentation finished")
+        else:
+            print("This part is not finished yet")
+    else:
+
+        print("Error, just train and validation code have written")
+
+def data_parser(args):
+
+    if args.model_state=='train':
+
+        train_files_name = args.train_list # dataset base dir
+
+        base_dir = os.path.join(args.dataset_dir, args.train_dataset,'edges') \
+            if args.train_dataset.lower() == 'biped' else os.path.join(args.dataset_dir, args.train_dataset)
+
+        train_list_path = os.path.join(base_dir, train_files_name)
+
+        train_list = read_files_list(train_list_path)
+
+        train_samples = split_pair_names(args,train_list, base_dir=base_dir)
+        n_train = len(train_samples)
+        print_info(" Enterely training set-up from {}, size: {}".format(train_list_path, n_train))
+
+        all_train_ids = np.arange(n_train)
+        np.random.shuffle(all_train_ids)
+
+        train_ids = all_train_ids[:int(args.train_split * len(train_list))]
+        valid_ids = all_train_ids[int(args.train_split * len(train_list)):]
+
+        print_info("Training set-up from {}, size: {}".format(train_list_path, len(train_ids)))
+        print_info("Validation set-up from {}, size: {}".format(train_list_path, len(valid_ids)))
+        cache_info = {
+            "files_path": train_samples,
+            "n_files": n_train,
+            "train_indices": train_ids,
+            "validation_indices": valid_ids
+        }
+        return cache_info
+
+# ************** for testing **********************
+    elif args.model_state=='test':
+        base_dir = os.path.join(args.dataset_dir,args.test_dataset,'edges')\
+            if args.test_dataset.upper()=='BIPED' else os.path.join(args.dataset_dir,args.test_dataset)
+
+        if args.test_dataset.upper() == "BIPED":
+            test_files_name = args.test_list
+            test_list_path = os.path.join(base_dir,test_files_name)
+            test_list = read_files_list(test_list_path)
+
+            test_samples = split_pair_names(args, test_list, base_dir)
+            n_test = len(test_samples)
+            print_info(" Enterely testing set-up from {}, size: {}".format(test_list_path, n_test))
+
+            test_ids = np.arange(n_test)
+            # np.random.shuffle(test_ids)
+
+            print_info("testing set-up from {}, size: {}".format(test_list_path, len(test_ids)))
+            cache_out = [test_samples, test_ids]
+
+            return cache_out
+
+        elif args.test_dataset == "BSDS":
+            test_files_name = args.test_list
+            test_list_path = os.path.join(args.dataset_dir + args.test_dataset,
+                                           test_files_name)
+            test_list = read_files_list(test_list_path)
+
+            test_samples = split_pair_names(args,test_list, args.dataset_dir + args.test_dataset)
+            n_test = len(test_samples)
+            print_info(" Enterely testing set-up from {}, size: {}".format(test_list_path, n_test))
+
+            test_ids = np.arange(n_test)
+            # np.random.shuffle(test_ids)
+
+            print_info("testing set-up from {}, size: {}".format(test_list_path, len(test_ids)))
+            cache_out = [test_samples, test_ids]
+            return cache_out
+
+        else:
+            # for NYUD
+            test_files_name = args.test_list
+            test_list_path = os.path.join(args.dataset_dir + args.test_dataset,
+                                          test_files_name)
+            test_list = read_files_list(test_list_path)
+
+            test_samples = split_pair_names(args,test_list, args.dataset_dir + args.test_dataset)
+            n_test = len(test_samples)
+            print_info(" Enterely testing set-up from {}, size: {}".format(test_list_path, n_test))
+
+            test_ids = np.arange(n_test)
+            # np.random.shuffle(test_ids)
+
+            print_info("testing set-up from {}, size: {}".format(test_list_path, len(test_ids)))
+            cache_out = [test_samples, test_ids]
+            return cache_out
+    else:
+        print_error("The model state is just train and test")
+        sys.exit()
+
+# ___________batch management ___________
+def get_batch(arg,file_list, batch=None, use_batch=True):
+
+    if use_batch:
+        file_names =[]
+        images=[]
+        edgemaps=[]
+        for idx, b in enumerate(batch):
+            x = cv.imread(file_list[b][0]) #  Image.open(file_list[b][0])
+            y = cv.imread(file_list[b][1]) #  Image.open(file_list[b][1])
+            if arg.model_state=='test':
+                pass
+            else:
+                x = cv.resize(x, dsize=(arg.image_width,arg.image_height)) # x.resize((arg.image_width, arg.image_height))
+                y = cv.resize(y,dsize=(arg.image_width, arg.image_height))# y.resize((arg.image_width, arg.image_height))
+            # pay attention here
+            x = np.array(x, dtype=np.float32)
+            # x = x[:, :, arg.channel_swap] # while using opencv it is not necessary
+            x -= arg.mean_pixel_values[0:3]
+
+            y = cv.cvtColor(y, cv.COLOR_BGR2GRAY)# np.array(y.convert('L'), dtype=np.float32)
+            y = np.array(y, dtype=np.float32)
+            if arg.train_dataset.lower()!='biped':
+                y[y < 107] = 0  # first nothing second <50 third <30
+                y[y >= 107] = 255.0
+            # else:
+                # y[y < 51] = 0  # first 100
+
+            if arg.target_regression:
+                bin_y = y/255.0
+            else:
+                bin_y = np.zeros_like(y)
+                bin_y[np.where(y)]=1
+
+            bin_y = bin_y if bin_y.ndim ==2 else bin_y[:,:,0]
+            bin_y = np.expand_dims(bin_y,axis=2)
+
+            images.append(x)
+            edgemaps.append(bin_y)
+            file_names.append(file_list[b])
+
+        return images, edgemaps, file_names
+
+    else:
+        # for testing re-coding is needed
+        if arg.test_dataset=='BIPED' and (arg.model_state=='test' and arg.use_nir):
+            x_nir = Image.open(file_list[0])
+            x_rgb = Image.open(file_list[1])
+            real_size = x_rgb.shape
+            # y = Image.open(file_list[2])
+            if not arg.image_width % 6 == 0 and arg.image_width > 1000:
+                x_nir = x_nir.resize((arg.image_width, arg.image_height)) #--
+                x_rgb = x_rgb.resize((arg.image_width, arg.image_height)) # --
+            else:
+                x_nir = x_nir.resize((arg.image_width, arg.image_height))
+                x_rgb = x_rgb.resize((arg.image_width, arg.image_height))
+
+            # y = y.resize((arg.image_width, arg.image_height))
+
+            x_nir = x_nir.convert("L")
+            # pay attention here
+            x_nir = np.array(x_nir, dtype=np.float32)
+            x_nir = np.expand_dims(x_nir, axis=2)
+            x_rgb = np.array(x_rgb, dtype=np.float32)
+
+            x_rgb = x_rgb[:, :, arg.channel_swap]
+            x = np.concatenate((x_rgb, x_nir), axis=2)
+            x -= arg.mean_pixel_values
+            # y = np.array(y.convert('L'), dtype=np.float32)
+            # if arg.target_regression:
+            #     bin_y = y / 255.0
+            # else:
+            #     bin_y = np.zeros_like(y)
+            #     bin_y[np.where(y)] = 1
+            #
+            # bin_y = bin_y if bin_y.ndim == 2 else bin_y[:, :, 0]
+            # bin_y = np.expand_dims(bin_y, axis=2)
+            images =x
+            edgemaps  = None
+            file_info = (file_list[2], real_size)
+
+        else:
+            x = cv.imread(file_list[0])
+            y = cv.imread(file_list[1])
+            real_size = x.shape
+
+            if arg.test_dataset.lower()=='biped' or arg.test_dataset.lower()=='multicue':
+                pass
+            else:
+                x = cv.resize(x, dsize=(arg.image_width, arg.image_height))
+                y = cv.resize(y, dsize=(arg.image_width, arg.image_height))
+            x = np.array(x, dtype=np.float32)
+            # x = x[:, :, arg.channel_swap] # while using opencv it is not necessary
+            x -= arg.mean_pixel_values[:-1]
+            y = cv.cvtColor(y, cv.COLOR_BGR2GRAY)
+            y = np.array(y, dtype=np.float32)
+            if arg.target_regression:
+                bin_y = y / 255.0
+            else:
+                bin_y = np.zeros_like(y)
+                bin_y[np.where(y)] = 1
+
+            bin_y = bin_y if bin_y.ndim == 2 else bin_y[:, :, 0]
+            bin_y = np.expand_dims(bin_y, axis=2)
+
+            images = x
+            edgemaps = bin_y
+            file_info = (file_list[1],real_size)
+        return images, edgemaps, file_info
+
+def get_training_batch(arg, data_ids):
+    train_ids = data_ids['train_indices']
+    file_list= data_ids['files_path']
+    batch_ids = np.random.choice(train_ids,arg.batch_size_train)
+
+    return get_batch(arg, file_list, batch_ids)
+
+def get_validation_batch(arg, data_ids):
+    if arg.use_nir:
+        valid_ids = data_ids['validation_indices']
+        file_list = data_ids['files_path']
+        batch_ids = np.random.choice(valid_ids, arg.batch_size_val)
+    else:
+        valid_ids = data_ids['validation_indices']
+        file_list= data_ids['files_path']
+        batch_ids = np.random.choice(valid_ids,arg.batch_size_val)
+    return get_batch(arg,file_list,batch_ids)
+
+def get_testing_batch(arg,list_ids, use_batch=True, i=None):
+    if use_batch:
+        test_ids = list_ids[1]
+        file_list = list_ids[0]
+        batch_ids = test_ids[i:i + arg.batch_size_test]
+        return get_batch(arg,file_list,batch_ids)
+    else:
+        return get_batch(arg, list_ids[0],list_ids[1], use_batch=False)
+
+def open_images(file_list):
+    if len(file_list)>2 and not len(file_list)==3:
+        imgs=[]
+        file_names = []
+        for i in range(len(file_list)):
+            tmp = Image.open(file_list[i])
+            imgs.append(tmp)
+            file_names.append(file_list[i])
+
+    elif len(file_list)>2 and len(file_list)==3:
+
+        imgs = Image.open(file_list[2])
+        file_names = file_list
+    else:
+        imgs = Image.open(file_list[1])
+        file_names= file_list
+
+    return imgs, file_names
+
+# for testing on single images
+def get_single_image(args,file_path=None):
+
+    if file_path is None:
+        imgs_name = ["CLASSIC", None]
+        img_dir = 'data' if args.test_dataset in imgs_name else os.path.join(args.dataset_dir, args.test_dataset)
+        file_list = os.listdir(img_dir)
+        data =[]
+        for i in file_list:
+            data.append(os.path.join(img_dir,i))
+        return data
+    else:
+        img = cv.imread(file_path)
+        h,w,c=img.shape
+        x = np.array(img, dtype=np.float32)
+        if h==args.image_height and w==args.image_width:
+            pass
+        else:
+            x = cv.resize(x, dsize=(args.image_width, args.image_height))
+        # x = x[:, :, arg.channel_swap] # while using opencv it is not necessary
+        x -= args.mean_pixel_values[:-1]
+        img_info = (file_path,img.shape)
+        return x, img_info
+
+def visualize_result(imgs_list, arg):
+    """
+    function for tensorflow results
+    :param imgs_list: a list of prediction, gt and input data
+    :param arg:
+    :return: one image with the whole of imgs_list data
+    """
+    n_imgs = len(imgs_list)
+    data_list =[]
+    for i in range(n_imgs):
+        tmp = imgs_list[i]
+        if len(tmp.shape)==3 or len(tmp.shape)==2:
+            if len(tmp.shape)==2:
+                tmp = np.uint8(image_normalization(tmp))
+                tmp=cv.cvtColor(tmp, cv.COLOR_GRAY2BGR)
+                tmp = cv.bitwise_not(tmp)
+            else:
+                if tmp.shape[-1]==3:
+                    tmp=restore_rgb([arg.channel_swap,arg.mean_pixel_values[:3]],tmp)
+                    tmp = np.uint8(image_normalization(tmp))
+                else:
+                    tmp = np.squeeze(tmp)
+                    tmp = np.uint8(image_normalization(tmp))
+                    tmp = cv.cvtColor(tmp, cv.COLOR_GRAY2BGR)
+                    tmp = cv.bitwise_not(tmp)
+        else:
+            tmp= np.squeeze(tmp)
+            if len(tmp.shape) == 2:
+                tmp = np.uint8(image_normalization(tmp))
+                tmp = cv.bitwise_not(tmp)
+                tmp = cv.cvtColor(tmp, cv.COLOR_GRAY2BGR)
+            else:
+                tmp = np.uint8(image_normalization(tmp))
+        data_list.append(tmp)
+    img = data_list[0]
+    if n_imgs % 2 == 0:
+        imgs = np.zeros((img.shape[0] * 2 + 10, img.shape[1] * (n_imgs // 2) + ((n_imgs // 2 - 1) * 5), 3))
+    else:
+        imgs = np.zeros((img.shape[0] * 2 + 10, img.shape[1] * ((1 + n_imgs) // 2) + ((n_imgs // 2) * 5), 3))
+        n_imgs += 1
+
+    k=0
+    imgs = np.uint8(imgs)
+    i_step = img.shape[0]+10
+    j_step = img.shape[1]+5
+    for i in range(2):
+        for j in range(n_imgs//2):
+            if k<len(data_list):
+                imgs[i*i_step:i*i_step+img.shape[0],j*j_step:j*j_step+img.shape[1],:]=data_list[k]
+                k+=1
+            else:
+                pass
+    return imgs

legacy/utls/losses.py

@@ -0,0 +1,47 @@
+
+import tensorflow as tf
+
+
+def sigmoid_cross_entropy_balanced(logits, label, name='cross_entrony_loss'):
+    """
+    Initially proposed in: 'Holistically-Nested Edge Detection (CVPR 15)'
+    Implements Equation [2] in https://arxiv.org/pdf/1504.06375.pdf
+    Compute edge pixels for each training sample and set as pos_weights to
+    tf.nn.weighted_cross_entropy_with_logits
+    """
+    y = tf.cast(label, tf.float32)
+
+    count_neg = tf.reduce_sum(1.-y)
+    count_pos  = tf.reduce_sum(y)
+    # Equation [2]
+    beta = count_neg / (count_neg + count_pos)
+
+    # Equation [2] divide by 1 - beta
+    pos_weight = beta / (1 - beta)
+    cost = tf.nn.weighted_cross_entropy_with_logits(logits=logits, targets=y, pos_weight=pos_weight)
+    # cost = tf.
+    # Multiply by 1 - beta
+    cost = tf.reduce_mean(cost * (1 - beta))
+
+    # check if image has no edge pixels return 0 else return complete error function
+    return tf.where(tf.equal(count_pos, 0.0), 0.0, cost, name=name)
+
+
+# def class_balanced_cross_entropy_with_logits(logits,label,name='class_ballanced_cross_entropy'):
+#
+#     # Initialy proposed in: 'Holistically-Nested Edge Detection (CVPR 15)'
+#     with tf.name_scope(name) as scope:
+#         logits= tf.cast(logits, tf.float32)
+#         label = tf.cast(label, tf.float32)
+#
+#         n_positives = tf.reduce_sum(label)
+#         n_negatives = tf.reduce_sum(1.0-label)
+#
+#         beta = n_negatives/(n_negatives+n_positives)
+#         pos_weight = beta / (1-beta)
+#         check_weight = tf.identity(beta,name='check')
+#
+#         cost = tf.nn.weighted_cross_entropy_with_logits(targets=label,logits=logits,pos_weight=pos_weight)
+#         loss = tf.reduce_mean((1-beta)*cost)
+#
+#         return tf.where(tf.equal(beta,1.0),0.0,loss)

legacy/utls/utls.py

@@ -0,0 +1,385 @@
+""" Utilities
+
+"""
+import numpy as np
+import time
+from termcolor import colored
+import os,glob
+import h5py
+
+def get_model_trained_name(sess=None,data_dir=None, last_update=True):
+
+    if sess is not None or data_dir is not None:
+        path = os.path.join(data_dir,'*.ckpt')
+        if last_update:
+            mfiles= glob.glob(path)
+            last_file=max(mfiles,key=os.path.getctime)
+            return os.path.join(path,last_file)
+        else:
+            print('Not performet yet')
+
+def gamma_correction(i, g,gamma=True):
+    """Gamma correction
+    This function is for gamma corrections and de-correction 0.4040 0.3030 0.6060
+    :param i: image data
+    :param g: gamma value
+    :param gamma: if true do gamma correction if does not degamma correction
+    :return:if gamma gamma corrected image else image without gamma correction
+    """
+    i = np.float32(i)
+    if gamma:
+        img=i**g
+    else:
+        img=i**(1/g)
+    return img
+
+def image_normalization(img, img_min=0, img_max=255):
+    """ Image normalization given a minimum and maximum
+
+    This is a typical image normalization function
+    where the minimum and maximum of the image is needed
+    source: https://en.wikipedia.org/wiki/Normalization_(image_processing)
+    :param img: an image could be gray scale or color
+    :param img_min:  for default is 0
+    :param img_max: for default is 255
+    :return: a normalized image given a scale
+    """
+    img = np.float32(img)
+    epsilon=1e-12 # whenever an inconsistent image
+    img = (img-np.min(img))*(img_max-img_min)/((np.max(img)-np.min(img))+epsilon)+img_min
+    return img
+
+
+def normalization_data_0255(data):
+    """
+    data normalization in 0 till 1 range
+    :param data:
+    :return:
+    """
+    epsilon = 1e-12
+    if not len(data.shape)==2:
+        n_imgs = data.shape[0]
+        # data = np.float32(data)
+        if data.shape[-1]==3 and len(data.shape)==3:
+            data = ((data - np.min(data)) * 255 / ((np.max(data) - np.min(data)) + epsilon))
+            # data = ((data - np.min(data)) * 254 / (np.max(data) - np.min(data)))+1
+
+        elif data.shape[-1]==3 and len(data.shape)==4:
+            for i in range(n_imgs):
+                img = data[i,...]
+                data[i,:,:,:] = ((img - np.min(img)) * 255 / ((np.max(img) - np.min(img))+epsilon))
+        # print("Data normalized with:", data.shape[-1], "channels")
+        elif data.shape[-1]>3 or len(data.shape)>=4:
+            print('error normalizin 0-255 line 30')
+        else:
+            print('error normalizin 0-255 line 32')
+
+        return data
+    elif len(data.shape)==2:
+        # data = ((data-np.min(data))*255/(np.max(data)-np.min(data)))
+        data = ((data-np.min(data))*255/((np.max(data) - np.min(data)) + epsilon))
+
+        return data
+    else:
+        print('error normalization 0-255')
+
+def normalization_data_01(data):
+    """
+    data normalization in 0 till 1 range
+    :param data:
+    :return:
+    """
+    epsilon = 1e-12
+    if np.sum(np.isnan(data))>0:
+        print('NaN detected before Normalization')
+        return 'variable has NaN values'
+    if len(data.shape)>3:
+        n_imgs = data.shape[0]
+        data = np.float32(data)
+        if data.shape[-1]==3:
+            for i in range(n_imgs):
+                img = data[i,:,:,:]
+                data[i,:,:,:] = ((img - np.min(img)) * 1 / ((np.max(img) - np.min(img))+epsilon))
+
+        elif data.shape[-1]==4:
+            print('it is a  little naive, check it in line 64 seg utils.py')
+            for i in range(n_imgs):
+                nir = data[i, :, :, -1]
+                nir = ((nir - np.min(nir)) * 1 / ((np.max(nir) - np.min(nir)) + epsilon))
+                img = data[i, :, :, 0:3]
+                img = ((img - np.min(img)) * 1 / ((np.max(img) - np.min(img)) + epsilon))
+                data[i, :, :, 0:3] = img
+                data[i, :, :, -1] = nir
+        elif data.shape[-1]==2:
+            #normalization according to channels
+            print('check line 70 utils_seg.py')
+            for i in range(n_imgs):
+                im = data[i,:,:,0]
+                N = data[i,:,:,-1]
+                data[i,:,:,0]= ((im-np.min(im))*1/(np.max(im)-np.min(im)))
+                data[i, :, :, -1] = ((N - np.min(N)) * 1 / (np.max(N) - np.min(N)))
+
+        elif data.shape[-1]==1:
+            for i in range(n_imgs):
+                img = data[i, :, :, 0]
+                data[i, :, :, 0] = ((img - np.min(img)) * 1 / ((np.max(img) - np.min(img))+epsilon))
+        else:
+            print("error normalizing line 83")
+        if np.sum(np.isnan(data)) > 0:
+            print('NaN detected after normalization')
+            return 'variable has NaN values'
+        return data
+
+    else:
+        if np.max(data) ==0 and np.min(data)==0:
+            return data
+        if np.sum(np.isnan(data)) > 0:
+            print('NaN detected before normalization')
+            return 'variable has NaN values'
+
+        data = ((data - np.min(data)) * 1 / ((np.max(data) - np.min(data))+epsilon))
+        if np.sum(np.isnan(data)) > 0:
+            print('NaN detected after normalization')
+            return 'variable has NaN values'
+        return data
+
+# _________ text visualization ____________
+def get_local_time():
+    return time.strftime("%d %b %Y %Hh%Mm%Ss", time.localtime())
+
+def print_info(info_string, quite=False):
+
+    info = '[{0}][INFO]{1}'.format(get_local_time(), info_string)
+    print(colored(info, 'green'))
+
+def print_error(error_string):
+
+    error = '[{0}][ERROR] {1}'.format(get_local_time(), error_string)
+    print (colored(error, 'red'))
+
+def print_warning(warning_string):
+
+    warning = '[{0}][WARNING] {1}'.format(get_local_time(), warning_string)
+
+    print (colored(warning, 'blue'))
+# ___________ End text visualization
+
+# ___________ read list of files
+def read_files_list(list_path,dataset_name=None):
+    mfiles = open(list_path)
+    file_names = mfiles.readlines()
+    mfiles.close()
+
+    file_names = [f.strip() for f in file_names]
+    return file_names
+
+def split_pair_names(opts, file_names, base_dir=None):
+    # If base_dir is None, it assume that the list have the complete image source
+    if opts.model_state=='train':
+
+        if base_dir==None:
+            file_names =[c.split(' ') for c in file_names]
+        else:
+            if opts.train_dataset.lower()=='biped':
+                x_base_dir=os.path.join(base_dir,'imgs',opts.model_state)
+                y_base_dir =os.path.join(base_dir,'edge_maps',opts.model_state)
+                file_names = [c.split(' ') for c in file_names]
+                file_names = [(os.path.join(x_base_dir, c[0]),
+                               os.path.join(y_base_dir, c[1])) for c in file_names]
+            else:
+                file_names = [c.split(' ') for c in file_names]
+                file_names = [(os.path.join(base_dir, c[0]),
+                               os.path.join(base_dir, c[1])) for c in file_names]
+        return file_names
+    else:
+        # ******************* for data testing ****************************
+        if base_dir == None:
+            file_names = [c.split(' ') for c in file_names]
+        else:
+            if opts.test_dataset.lower() == 'biped':
+                x_base_dir = os.path.join(base_dir,'imgs', opts.model_state)
+                y_base_dir = os.path.join(base_dir,'edge_maps', opts.model_state)
+                file_names = [c.split(' ') for c in file_names]
+                file_names = [(os.path.join(x_base_dir, c[0]),
+                               os.path.join(y_base_dir, c[1])) for c in file_names]
+            else:
+                file_names = [c.split(' ') for c in file_names]
+                file_names = [(os.path.join(base_dir, c[0]),
+                               os.path.join(base_dir, c[1])) for c in file_names]
+
+        return file_names
+# ____________ End reading files list
+
+# _____________ H5 file manager _________
+
+def h5_reader(path):
+    """ Read H5 file
+    Read .h5 file format data h5py <<.File>>
+    :param path:file path of desired file
+    :return: dataset -> contain images data for training;
+    label -> contain  training label values (ground truth)
+    """
+    with h5py.File(path, 'r') as hf:
+        n_variables = len(list(hf.keys()))
+        # choice = True  # write
+        if n_variables==3:
+            data = np.array(hf.get('data'))
+            label = np.array(hf.get('label'))
+            test = np.array(hf.get('test'))
+        elif n_variables==2:
+
+            data = np.array(hf.get('data'))
+            label = np.array(hf.get('label'))
+            test=None
+        elif n_variables == 1:
+            data = np.array(hf.get('data'))
+            label=None
+            test=None
+        else:
+            data = None
+            label = None
+            test = None
+            print("Error reading path: ",path)
+
+        print(n_variables, " vars opened from: ", path)
+        return data, label, test
+
+def save_h5_data(savepath,data, label, predi = None, data_name=None,
+                 label_name=None, predi_name=None):
+    if data_name==None or label_name==None:
+        if np.any(predi == None):
+
+            with h5py.File(savepath, 'w') as hf:
+                hf.create_dataset('data', data=data)
+                hf.create_dataset('label', data=label)
+                print("Data [", data.shape, "and label ", label.shape, "] saved in: ", savepath)
+        else:
+            with h5py.File(savepath, 'w') as hf:
+                hf.create_dataset('data', data=data)
+                hf.create_dataset('label', data=label)
+                hf.create_dataset('predi', data=predi)
+                print("Input data [", data.shape, ", label ", label.shape, "and predi ", predi.shape,"] saved in: ", savepath)
+
+
+    else:
+        with h5py.File(savepath, 'w') as hf:
+            hf.create_dataset(data_name, data=data)
+            hf.create_dataset(label_name, data=label)
+            hf.create_dataset(predi_name,data=predi)
+            print("[",data_name, data.shape, ", ",
+                  label_name, label.shape," and ",predi_name, predi.shape, "] saved in: ", savepath)
+
+def save_variable_h5(savepath, data):
+
+    with h5py.File(savepath, 'w') as hf:
+        hf.create_dataset('data', data=data)
+
+        print("Data [", len(data), "] saved in: ", savepath)
+# ___________ End h5 file manager ____________
+
+# ____________ Restoring RGB former values _______
+
+def restore_rgb(config,I):
+    """
+    :param config: [args.channel_swap, args.mean_pixel_value]
+    :param I: and image or a set of images
+    :return: an image or a set of images restored
+    """
+    if  len(I)>3 and not type(I)==np.ndarray:
+        I =np.array(I)
+        I = I[:,:,:,0:3]
+        n = I.shape[0]
+        for i in range(n):
+            x = I[i,...]
+            x = np.array(x, dtype=np.float32)
+            x += config[1]
+            x = x[:, :, config[0]]
+            x = normalization_data_0255(x)
+            I[i,:,:,:]=x
+    elif len(I.shape)==3 and I.shape[-1]==3:
+        I = np.array(I, dtype=np.float32)
+        I += config[1]
+        I = I[:, :, config[0]]
+        I = normalization_data_0255(I)
+    else:
+        print_error("Sorry the input data size is out of our configuration")
+    print_info("The enterely I data {} restored".format(I.shape))
+    return I
+
+def restore_edgemap(config,I):
+    """ Not finished coding ***
+    :param config: args.target_regression = True or False
+    :param I:  input image data
+    :return: restored image data
+    """
+    print_error("Sorry this function is not ready")
+    if len(I.shape)>3 and I.shape[3]==1:
+        n = I.shape[0]
+        for i in range(n):
+            y=I[i,...]
+    elif len(I.shape)==3 and I.shape[-1]==1:
+
+        I = np.array(I.convert('L'), dtype=np.float32)
+        if config:
+            bin_I = I / 255.0
+        else:
+            bin_I = np.zeros_like(I)
+            bin_I[np.where(I)] = 1
+
+        bin_I = bin_I if bin_I.ndim == 2 else bin_I[:, :, 0]
+        bin_y = np.expand_dims(bin_I, axis=2)
+    else:
+        print_error("Sorry the input data size is out of our configuration")
+    return I
+
+def tensor_norm_01(data):
+    """
+    tensor means that the size image is [batch-size,img_width, img_height, num_channels]
+    :param data:
+    :return:
+    """
+    data = np.array(data)
+    if np.sum(np.isnan(data))>0:
+        print('NaN detected before Normalization')
+        return 'variable has NaN values'
+    if len(data.shape)>3:
+        n_imgs = data.shape[0]
+        data = np.float32(data)
+        if data.shape[-1]==3:
+            for i in range(n_imgs):
+                img = data[i,:,:,:]
+                data[i,:,:,:] = image_normalization(img,img_min=0,img_max=1)
+
+        elif data.shape[-1]==4:
+            print('it is a  little naive, check it in line 64 seg utils.py')
+            for i in range(n_imgs):
+                nir = data[i, :, :, -1]
+                nir = image_normalization(nir,img_min=0,img_max=1)
+                img = data[i, :, :, 0:3]
+                img = image_normalization(img,img_min=0,img_max=1)
+                data[i, :, :, 0:3] = img
+                data[i, :, :, -1] = nir
+        elif data.shape[-1]==2:
+            #normalization according to channels
+            print('check line 70 utils_seg.py')
+            for i in range(n_imgs):
+                im = data[i,:,:,0]
+                N = data[i,:,:,-1]
+                data[i,:,:,0]= image_normalization(im,img_min=0,img_max=1)
+                data[i, :, :, -1] = image_normalization(N,img_min=0,img_max=1)
+
+        elif data.shape[-1]==1:
+            x=[]
+            for i in range(n_imgs):
+                img = data[i, :, :, 0]
+                img= image_normalization(img,img_min=0,img_max=1)
+                x.append(img)
+            data=x
+        else:
+            print("error normalizing line 83")
+        if np.sum(np.isnan(data)) > 0:
+            print('NaN detected after normalization')
+            return 'variable has NaN values'
+        return data
+    else:
+        print('Please use image_normalization() function')

losses.py

@@ -0,0 +1,148 @@
+import torch
+import torch.nn.functional as F
+from dexi_utils import *
+
+
+def hed_loss2(inputs, targets, l_weight=1.1):
+    # bdcn loss with the rcf approach
+    targets = targets.long()
+    mask = targets.float()
+    num_positive = torch.sum((mask > 0.1).float()).float()
+    num_negative = torch.sum((mask <= 0.).float()).float()
+
+    mask[mask > 0.1] = 1.0 * num_negative / (num_positive + num_negative)
+    mask[mask <= 0.] = 1.1 * num_positive / (num_positive + num_negative)
+    inputs= torch.sigmoid(inputs)
+    cost = torch.nn.BCELoss(mask, reduction='sum')(inputs.float(), targets.float())
+
+    return l_weight*torch.sum(cost)
+
+
+def bdcn_loss2(inputs, targets, l_weight=1.1):
+    # bdcn loss with the rcf approach
+    targets = targets.long()
+    # mask = (targets > 0.1).float()
+    mask = targets.float()
+    num_positive = torch.sum((mask > 0.0).float()).float() # >0.1
+    num_negative = torch.sum((mask <= 0.0).float()).float() # <= 0.1
+
+    mask[mask > 0.] = 1.0 * num_negative / (num_positive + num_negative) #0.1
+    mask[mask <= 0.] = 1.1 * num_positive / (num_positive + num_negative)  # before mask[mask <= 0.1]
+    # mask[mask == 2] = 0
+    inputs= torch.sigmoid(inputs)
+    cost = torch.nn.BCELoss(mask, reduction='none')(inputs, targets.float())
+    # cost = torch.mean(cost.float().mean((1, 2, 3))) # before sum
+    cost = torch.sum(cost.float().mean((1, 2, 3))) # before sum
+    return l_weight*cost
+
+def bdcn_lossORI(inputs, targets, l_weigts=1.1,cuda=False):
+    """
+    :param inputs: inputs is a 4 dimensional data nx1xhxw
+    :param targets: targets is a 3 dimensional data nx1xhxw
+    :return:
+    """
+    n, c, h, w = inputs.size()
+    # print(cuda)
+    weights = np.zeros((n, c, h, w))
+    for i in range(n):
+        t = targets[i, :, :, :].cpu().data.numpy()
+        pos = (t == 1).sum()
+        neg = (t == 0).sum()
+        valid = neg + pos
+        weights[i, t == 1] = neg * 1. / valid
+        weights[i, t == 0] = pos * 1.1 / valid  # balance = 1.1
+    weights = torch.Tensor(weights)
+    # if cuda:
+    weights = weights.cuda()
+    inputs = torch.sigmoid(inputs)
+    loss = torch.nn.BCELoss(weights, reduction='sum')(inputs.float(), targets.float())
+    return l_weigts*loss
+
+def rcf_loss(inputs, label):
+
+    label = label.long()
+    mask = label.float()
+    num_positive = torch.sum((mask > 0.5).float()).float() # ==1.
+    num_negative = torch.sum((mask == 0).float()).float()
+
+    mask[mask == 1] = 1.0 * num_negative / (num_positive + num_negative)
+    mask[mask == 0] = 1.1 * num_positive / (num_positive + num_negative)
+    mask[mask == 2] = 0.
+    inputs= torch.sigmoid(inputs)
+    cost = torch.nn.BCELoss(mask, reduction='sum')(inputs.float(), label.float())
+
+    return 1.*torch.sum(cost)
+
+# ------------ cats losses ----------
+
+def bdrloss(prediction, label, radius,device='cpu'):
+    '''
+    The boundary tracing loss that handles the confusing pixels.
+    '''
+
+    filt = torch.ones(1, 1, 2*radius+1, 2*radius+1)
+    filt.requires_grad = False
+    filt = filt.to(device)
+
+    bdr_pred = prediction * label
+    pred_bdr_sum = label * F.conv2d(bdr_pred, filt, bias=None, stride=1, padding=radius)
+    texture_mask = F.conv2d(label.float(), filt, bias=None, stride=1, padding=radius)
+    mask = (texture_mask != 0).float()
+    mask[label == 1] = 0
+    pred_texture_sum = F.conv2d(prediction * (1-label) * mask, filt, bias=None, stride=1, padding=radius)
+
+    softmax_map = torch.clamp(pred_bdr_sum / (pred_texture_sum + pred_bdr_sum + 1e-10), 1e-10, 1 - 1e-10)
+    cost = -label * torch.log(softmax_map)
+    cost[label == 0] = 0
+
+    return cost.sum()
+
+
+
+def textureloss(prediction, label, mask_radius, device='cpu'):
+    '''
+    The texture suppression loss that smooths the texture regions.
+    '''
+    filt1 = torch.ones(1, 1, 3, 3)
+    filt1.requires_grad = False
+    filt1 = filt1.to(device)
+    filt2 = torch.ones(1, 1, 2*mask_radius+1, 2*mask_radius+1)
+    filt2.requires_grad = False
+    filt2 = filt2.to(device)
+
+    pred_sums = F.conv2d(prediction.float(), filt1, bias=None, stride=1, padding=1)
+    label_sums = F.conv2d(label.float(), filt2, bias=None, stride=1, padding=mask_radius)
+
+    mask = 1 - torch.gt(label_sums, 0).float()
+
+    loss = -torch.log(torch.clamp(1-pred_sums/9, 1e-10, 1-1e-10))
+    loss[mask == 0] = 0
+
+    return torch.sum(loss)
+
+
+def cats_loss(prediction, label, l_weight=[0.,0.], device='cpu'):
+    # tracingLoss
+    tex_factor,bdr_factor = l_weight
+    balanced_w = 1.1
+    label = label.float()
+    prediction = prediction.float()
+    with torch.no_grad():
+        mask = label.clone()
+
+        num_positive = torch.sum((mask == 1).float()).float()
+        num_negative = torch.sum((mask == 0).float()).float()
+        beta = num_negative / (num_positive + num_negative)
+        mask[mask == 1] = beta
+        mask[mask == 0] = balanced_w * (1 - beta)
+        mask[mask == 2] = 0
+    prediction = torch.sigmoid(prediction)
+    # print('bce')
+    cost = torch.sum(torch.nn.functional.binary_cross_entropy(
+        prediction.float(), label.float(), weight=mask, reduce=False))
+    label_w = (label != 0).float()
+    # print('tex')
+    textcost = textureloss(prediction.float(), label_w.float(), mask_radius=4, device=device)
+    bdrcost = bdrloss(prediction.float(), label_w.float(), radius=4, device=device)
+
+    return cost + bdr_factor * bdrcost + tex_factor * textcost

main.py

@@ -0,0 +1,481 @@
+
+from __future__ import print_function
+
+import argparse
+import os
+import time, platform
+
+import cv2
+import torch.optim as optim
+from torch.utils.data import DataLoader
+
+from datasets import DATASET_NAMES, BipedDataset, TestDataset, dataset_info
+from losses import *
+from model import DexiNed
+from utils import (image_normalization, save_image_batch_to_disk,
+                   visualize_result,count_parameters)
+
+IS_LINUX = True if platform.system()=="Linux" else False
+def train_one_epoch(epoch, dataloader, model, criterion, optimizer, device,
+                    log_interval_vis, tb_writer, args=None):
+    imgs_res_folder = os.path.join(args.output_dir, 'current_res')
+    os.makedirs(imgs_res_folder,exist_ok=True)
+
+    # Put model in training mode
+    model.train()
+    # l_weight = [0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 1.1]  # for bdcn ori loss
+     # before [0.6,0.6,1.1,1.1,0.4,0.4,1.3] [0.4,0.4,1.1,1.1,0.6,0.6,1.3],[0.4,0.4,1.1,1.1,0.8,0.8,1.3]
+    l_weight = [0.7,0.7,1.1,1.1,0.3,0.3,1.3] # New BDCN  loss
+    # l_weight = [[0.05, 2.], [0.05, 2.], [0.05, 2.],
+    #             [0.1, 1.], [0.1, 1.], [0.1, 1.],
+    #             [0.01, 4.]]  # for cats loss
+    loss_avg =[]
+    for batch_id, sample_batched in enumerate(dataloader):
+        images = sample_batched['images'].to(device)  # BxCxHxW
+        labels = sample_batched['labels'].to(device)  # BxHxW
+        preds_list = model(images)
+        # loss = sum([criterion(preds, labels, l_w, device) for preds, l_w in zip(preds_list, l_weight)])  # cats_loss
+        loss = sum([criterion(preds, labels,l_w) for preds, l_w in zip(preds_list,l_weight)]) # bdcn_loss
+        # loss = sum([criterion(preds, labels) for preds in preds_list])  #HED loss, rcf_loss
+        optimizer.zero_grad()
+        loss.backward()
+        optimizer.step()
+        loss_avg.append(loss.item())
+        if epoch==0 and (batch_id==100 and tb_writer is not None):
+            tmp_loss = np.array(loss_avg).mean()
+            tb_writer.add_scalar('loss', tmp_loss,epoch)
+
+        if batch_id % 5 == 0:
+            print(time.ctime(), 'Epoch: {0} Sample {1}/{2} Loss: {3}'
+                  .format(epoch, batch_id, len(dataloader), loss.item()))
+        if batch_id % log_interval_vis == 0:
+            res_data = []
+
+            img = images.cpu().numpy()
+            res_data.append(img[2])
+
+            ed_gt = labels.cpu().numpy()
+            res_data.append(ed_gt[2])
+
+            # tmp_pred = tmp_preds[2,...]
+            for i in range(len(preds_list)):
+                tmp = preds_list[i]
+                tmp = tmp[2]
+                # print(tmp.shape)
+                tmp = torch.sigmoid(tmp).unsqueeze(dim=0)
+                tmp = tmp.cpu().detach().numpy()
+                res_data.append(tmp)
+
+            vis_imgs = visualize_result(res_data, arg=args)
+            del tmp, res_data
+
+            vis_imgs = cv2.resize(vis_imgs,
+                                  (int(vis_imgs.shape[1]*0.8), int(vis_imgs.shape[0]*0.8)))
+            img_test = 'Epoch: {0} Sample {1}/{2} Loss: {3}' \
+                .format(epoch, batch_id, len(dataloader), loss.item())
+
+            BLACK = (0, 0, 255)
+            font = cv2.FONT_HERSHEY_SIMPLEX
+            font_size = 1.1
+            font_color = BLACK
+            font_thickness = 2
+            x, y = 30, 30
+            vis_imgs = cv2.putText(vis_imgs,
+                                   img_test,
+                                   (x, y),
+                                   font, font_size, font_color, font_thickness, cv2.LINE_AA)
+            cv2.imwrite(os.path.join(imgs_res_folder, 'results.png'), vis_imgs)
+    loss_avg = np.array(loss_avg).mean()
+    return loss_avg
+
+
+def validate_one_epoch(epoch, dataloader, model, device, output_dir, arg=None):
+    # XXX This is not really validation, but testing
+
+    # Put model in eval mode
+    model.eval()
+
+    with torch.no_grad():
+        for _, sample_batched in enumerate(dataloader):
+            images = sample_batched['images'].to(device)
+            # labels = sample_batched['labels'].to(device)
+            file_names = sample_batched['file_names']
+            image_shape = sample_batched['image_shape']
+            preds = model(images)
+            # print('pred shape', preds[0].shape)
+            save_image_batch_to_disk(preds[-1],
+                                     output_dir,
+                                     file_names,img_shape=image_shape,
+                                     arg=arg)
+
+
+def test(checkpoint_path, dataloader, model, device, output_dir, args):
+    if not os.path.isfile(checkpoint_path):
+        raise FileNotFoundError(
+            f"Checkpoint filte note found: {checkpoint_path}")
+    print(f"Restoring weights from: {checkpoint_path}")
+    model.load_state_dict(torch.load(checkpoint_path,
+                                     map_location=device))
+
+    # Put model in evaluation mode
+    model.eval()
+
+    with torch.no_grad():
+        total_duration = []
+        for batch_id, sample_batched in enumerate(dataloader):
+            images = sample_batched['images'].to(device)
+            if not args.test_data == "CLASSIC":
+                labels = sample_batched['labels'].to(device)
+            file_names = sample_batched['file_names']
+            image_shape = sample_batched['image_shape']
+            print(f"input tensor shape: {images.shape}")
+            # images = images[:, [2, 1, 0], :, :]
+
+            end = time.perf_counter()
+            if device.type == 'cuda':
+                torch.cuda.synchronize()
+            preds = model(images)
+            if device.type == 'cuda':
+                torch.cuda.synchronize()
+            tmp_duration = time.perf_counter() - end
+            total_duration.append(tmp_duration)
+
+            save_image_batch_to_disk(preds,
+                                     output_dir,
+                                     file_names,
+                                     image_shape,
+                                     arg=args)
+            torch.cuda.empty_cache()
+
+    total_duration = np.sum(np.array(total_duration))
+    print("******** Testing finished in", args.test_data, "dataset. *****")
+    print("FPS: %f.4" % (len(dataloader)/total_duration))
+
+def testPich(checkpoint_path, dataloader, model, device, output_dir, args):
+    # a test model plus the interganged channels
+    if not os.path.isfile(checkpoint_path):
+        raise FileNotFoundError(
+            f"Checkpoint filte note found: {checkpoint_path}")
+    print(f"Restoring weights from: {checkpoint_path}")
+    model.load_state_dict(torch.load(checkpoint_path,
+                                     map_location=device))
+
+    # Put model in evaluation mode
+    model.eval()
+
+    with torch.no_grad():
+        total_duration = []
+        for batch_id, sample_batched in enumerate(dataloader):
+            images = sample_batched['images'].to(device)
+            if not args.test_data == "CLASSIC":
+                labels = sample_batched['labels'].to(device)
+            file_names = sample_batched['file_names']
+            image_shape = sample_batched['image_shape']
+            print(f"input tensor shape: {images.shape}")
+            start_time = time.time()
+            # images2 = images[:, [1, 0, 2], :, :]  #GBR
+            images2 = images[:, [2, 1, 0], :, :] # RGB
+            preds = model(images)
+            preds2 = model(images2)
+            tmp_duration = time.time() - start_time
+            total_duration.append(tmp_duration)
+            save_image_batch_to_disk([preds,preds2],
+                                     output_dir,
+                                     file_names,
+                                     image_shape,
+                                     arg=args, is_inchannel=True)
+            torch.cuda.empty_cache()
+
+    total_duration = np.array(total_duration)
+    print("******** Testing finished in", args.test_data, "dataset. *****")
+    print("Average time per image: %f.4" % total_duration.mean(), "seconds")
+    print("Time spend in the Dataset: %f.4" % total_duration.sum(), "seconds")
+
+
+def parse_args():
+    """Parse command line arguments."""
+    parser = argparse.ArgumentParser(description='DexiNed trainer.')
+    parser.add_argument('--choose_test_data',
+                        type=int,
+                        default=-1,
+                        help='Already set the dataset for testing choice: 0 - 8')
+    # ----------- test -------0--
+
+
+    TEST_DATA = DATASET_NAMES[parser.parse_args().choose_test_data] # max 8
+    test_inf = dataset_info(TEST_DATA, is_linux=IS_LINUX)
+    test_dir = test_inf['data_dir']
+    is_testing =True#  current test -352-SM-NewGT-2AugmenPublish
+
+    # Training settings
+    TRAIN_DATA = DATASET_NAMES[0] # BIPED=0, MDBD=6
+    train_inf = dataset_info(TRAIN_DATA, is_linux=IS_LINUX)
+    train_dir = train_inf['data_dir']
+
+
+    # Data parameters
+    parser.add_argument('--input_dir',
+                        type=str,
+                        default=train_dir,
+                        help='the path to the directory with the input data.')
+    parser.add_argument('--input_val_dir',
+                        type=str,
+                        default=test_inf['data_dir'],
+                        help='the path to the directory with the input data for validation.')
+    parser.add_argument('--output_dir',
+                        type=str,
+                        default='checkpoints',
+                        help='the path to output the results.')
+    parser.add_argument('--train_data',
+                        type=str,
+                        choices=DATASET_NAMES,
+                        default=TRAIN_DATA,
+                        help='Name of the dataset.')
+    parser.add_argument('--test_data',
+                        type=str,
+                        choices=DATASET_NAMES,
+                        default=TEST_DATA,
+                        help='Name of the dataset.')
+    parser.add_argument('--test_list',
+                        type=str,
+                        default=test_inf['test_list'],
+                        help='Dataset sample indices list.')
+    parser.add_argument('--train_list',
+                        type=str,
+                        default=train_inf['train_list'],
+                        help='Dataset sample indices list.')
+    parser.add_argument('--is_testing',type=bool,
+                        default=is_testing,
+                        help='Script in testing mode.')
+    parser.add_argument('--double_img',
+                        type=bool,
+                        default=False,
+                        help='True: use same 2 imgs changing channels')  # Just for test
+    parser.add_argument('--resume',
+                        type=bool,
+                        default=False,
+                        help='use previous trained data')  # Just for test
+    parser.add_argument('--checkpoint_data',
+                        type=str,
+                        default='10/10_model.pth',# 4 6 7 9 14
+                        help='Checkpoint path from which to restore model weights from.')
+    parser.add_argument('--test_img_width',
+                        type=int,
+                        default=test_inf['img_width'],
+                        help='Image width for testing.')
+    parser.add_argument('--test_img_height',
+                        type=int,
+                        default=test_inf['img_height'],
+                        help='Image height for testing.')
+    parser.add_argument('--res_dir',
+                        type=str,
+                        default='result',
+                        help='Result directory')
+    parser.add_argument('--log_interval_vis',
+                        type=int,
+                        default=50,
+                        help='The number of batches to wait before printing test predictions.')
+
+    parser.add_argument('--epochs',
+                        type=int,
+                        default=17,
+                        metavar='N',
+                        help='Number of training epochs (default: 25).')
+    parser.add_argument('--lr',
+                        default=1e-4,
+                        type=float,
+                        help='Initial learning rate.')
+    parser.add_argument('--wd',
+                        type=float,
+                        default=1e-8,
+                        metavar='WD',
+                        help='weight decay (Good 1e-8) in TF1=0') # 1e-8 -> BIRND/MDBD, 0.0 -> BIPED
+    parser.add_argument('--adjust_lr',
+                        default=[10,15],
+                        type=int,
+                        help='Learning rate step size.') #[5,10]BIRND [10,15]BIPED/BRIND
+    parser.add_argument('--batch_size',
+                        type=int,
+                        default=8,
+                        metavar='B',
+                        help='the mini-batch size (default: 8)')
+    parser.add_argument('--workers',
+                        default=16,
+                        type=int,
+                        help='The number of workers for the dataloaders.')
+    parser.add_argument('--tensorboard',type=bool,
+                        default=True,
+                        help='Use Tensorboard for logging.'),
+    parser.add_argument('--img_width',
+                        type=int,
+                        default=352,
+                        help='Image width for training.') # BIPED 400 BSDS 352/320 MDBD 480
+    parser.add_argument('--img_height',
+                        type=int,
+                        default=352,
+                        help='Image height for training.') # BIPED 480 BSDS 352/320
+    parser.add_argument('--channel_swap',
+                        default=[2, 1, 0],
+                        type=int)
+    parser.add_argument('--crop_img',
+                        default=True,
+                        type=bool,
+                        help='If true crop training images, else resize images to match image width and height.')
+    parser.add_argument('--mean_pixel_values',
+                        default=[103.939,116.779,123.68, 137.86],
+                        type=float)  # [103.939,116.779,123.68] [104.00699, 116.66877, 122.67892]
+    args = parser.parse_args()
+    return args
+
+
+def main(args):
+    """Main function."""
+
+    print(f"Number of GPU's available: {torch.cuda.device_count()}")
+    print(f"Pytorch version: {torch.__version__}")
+
+    # Tensorboard summary writer
+
+    tb_writer = None
+    training_dir = os.path.join(args.output_dir,args.train_data)
+    os.makedirs(training_dir,exist_ok=True)
+    checkpoint_path = os.path.join(args.output_dir, args.train_data, args.checkpoint_data)
+    if args.tensorboard and not args.is_testing:
+        from torch.utils.tensorboard import SummaryWriter # for torch 1.4 or greather
+        tb_writer = SummaryWriter(log_dir=training_dir)
+        # saving Model training settings
+        training_notes = ['DexiNed, Xavier Normal Init, LR= ' + str(args.lr) + ' WD= '
+                          + str(args.wd) + ' image size = ' + str(args.img_width)
+                          + ' adjust LR='+ str(args.adjust_lr) + ' Loss Function= BDCNloss2. '
+                          +'Trained on> '+args.train_data+' Tested on> '
+                          +args.test_data+' Batch size= '+str(args.batch_size)+' '+str(time.asctime())]
+        info_txt = open(os.path.join(training_dir, 'training_settings.txt'), 'w')
+        info_txt.write(str(training_notes))
+        info_txt.close()
+
+    # Get computing device
+    device = torch.device('cpu' if torch.cuda.device_count() == 0
+                          else 'cuda')
+
+    # Instantiate model and move it to the computing device
+    model = DexiNed().to(device)
+    # model = nn.DataParallel(model)
+    ini_epoch =0
+    if not args.is_testing:
+        if args.resume:
+            ini_epoch=11
+            model.load_state_dict(torch.load(checkpoint_path,
+                                         map_location=device))
+            print('Training restarted from> ',checkpoint_path)
+        dataset_train = BipedDataset(args.input_dir,
+                                     img_width=args.img_width,
+                                     img_height=args.img_height,
+                                     mean_bgr=args.mean_pixel_values[0:3] if len(
+                                         args.mean_pixel_values) == 4 else args.mean_pixel_values,
+                                     train_mode='train',
+                                     arg=args
+                                     )
+        dataloader_train = DataLoader(dataset_train,
+                                      batch_size=args.batch_size,
+                                      shuffle=True,
+                                      num_workers=args.workers)
+
+    dataset_val = TestDataset(args.input_val_dir,
+                              test_data=args.test_data,
+                              img_width=args.test_img_width,
+                              img_height=args.test_img_height,
+                              mean_bgr=args.mean_pixel_values[0:3] if len(
+                                  args.mean_pixel_values) == 4 else args.mean_pixel_values,
+                              test_list=args.test_list, arg=args
+                              )
+    dataloader_val = DataLoader(dataset_val,
+                                batch_size=1,
+                                shuffle=False,
+                                num_workers=args.workers)
+    # Testing
+    if args.is_testing:
+
+        output_dir = os.path.join(args.res_dir, args.train_data+"2"+ args.test_data)
+        print(f"output_dir: {output_dir}")
+        if args.double_img:
+            # predict twice an image changing channels, then mix those results
+            testPich(checkpoint_path, dataloader_val, model, device, output_dir, args)
+        else:
+            test(checkpoint_path, dataloader_val, model, device, output_dir, args)
+
+        num_param = count_parameters(model)
+        print('-------------------------------------------------------')
+        print('DexiNed # of Parameters:')
+        print(num_param)
+        print('-------------------------------------------------------')
+        return
+
+    criterion = bdcn_loss2 # hed_loss2 #bdcn_loss2
+
+    optimizer = optim.Adam(model.parameters(),
+                           lr=args.lr,
+                           weight_decay=args.wd)
+
+    # Main training loop
+    seed=1021
+    adjust_lr = args.adjust_lr
+    lr2= args.lr
+    for epoch in range(ini_epoch,args.epochs):
+        if epoch%7==0:
+
+            seed = seed+1000
+            np.random.seed(seed)
+            torch.manual_seed(seed)
+            torch.cuda.manual_seed(seed)
+            print("------ Random seed applied-------------")
+        # Create output directories
+        if adjust_lr is not None:
+            if epoch in adjust_lr:
+                lr2 = lr2*0.1
+                for param_group in optimizer.param_groups:
+                    param_group['lr'] = lr2
+
+        output_dir_epoch = os.path.join(args.output_dir,args.train_data, str(epoch))
+        img_test_dir = os.path.join(output_dir_epoch, args.test_data + '_res')
+        os.makedirs(output_dir_epoch,exist_ok=True)
+        os.makedirs(img_test_dir,exist_ok=True)
+        validate_one_epoch(epoch,
+                           dataloader_val,
+                           model,
+                           device,
+                           img_test_dir,
+                           arg=args)
+
+        avg_loss =train_one_epoch(epoch,
+                        dataloader_train,
+                        model,
+                        criterion,
+                        optimizer,
+                        device,
+                        args.log_interval_vis,
+                        tb_writer,
+                        args=args)
+        validate_one_epoch(epoch,
+                           dataloader_val,
+                           model,
+                           device,
+                           img_test_dir,
+                           arg=args)
+
+        # Save model after end of every epoch
+        torch.save(model.module.state_dict() if hasattr(model, "module") else model.state_dict(),
+                   os.path.join(output_dir_epoch, '{0}_model.pth'.format(epoch)))
+        if tb_writer is not None:
+            tb_writer.add_scalar('loss',
+                                 avg_loss,
+                                 epoch+1)
+        print('Current learning rate> ', optimizer.param_groups[0]['lr'])
+    num_param = count_parameters(model)
+    print('-------------------------------------------------------')
+    print('DexiNed, # of Parameters:')
+    print(num_param)
+    print('-------------------------------------------------------')
+
+if __name__ == '__main__':
+    args = parse_args()
+    main(args)

model.py

@@ -0,0 +1,286 @@
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+
+def weight_init(m):
+    if isinstance(m, (nn.Conv2d,)):
+        # torch.nn.init.xavier_uniform_(m.weight, gain=1.0)
+        torch.nn.init.xavier_normal_(m.weight, gain=1.0)
+        # torch.nn.init.normal_(m.weight, mean=0.0, std=0.01)
+        if m.weight.data.shape[1] == torch.Size([1]):
+            torch.nn.init.normal_(m.weight, mean=0.0)
+
+        if m.bias is not None:
+            torch.nn.init.zeros_(m.bias)
+
+    # for fusion layer
+    if isinstance(m, (nn.ConvTranspose2d,)):
+        # torch.nn.init.xavier_uniform_(m.weight, gain=1.0)
+        torch.nn.init.xavier_normal_(m.weight, gain=1.0)
+        # torch.nn.init.normal_(m.weight, mean=0.0, std=0.01)
+
+        if m.weight.data.shape[1] == torch.Size([1]):
+            torch.nn.init.normal_(m.weight, std=0.1)
+        if m.bias is not None:
+            torch.nn.init.zeros_(m.bias)
+
+
+class CoFusion(nn.Module):
+
+    def __init__(self, in_ch, out_ch):
+        super(CoFusion, self).__init__()
+        self.conv1 = nn.Conv2d(in_ch, 64, kernel_size=3,
+                               stride=1, padding=1)
+        self.conv2 = nn.Conv2d(64, 64, kernel_size=3,
+                               stride=1, padding=1)
+        self.conv3 = nn.Conv2d(64, out_ch, kernel_size=3,
+                               stride=1, padding=1)
+        self.relu = nn.ReLU()
+
+        self.norm_layer1 = nn.GroupNorm(4, 64)
+        self.norm_layer2 = nn.GroupNorm(4, 64)
+
+    def forward(self, x):
+        # fusecat = torch.cat(x, dim=1)
+        attn = self.relu(self.norm_layer1(self.conv1(x)))
+        attn = self.relu(self.norm_layer2(self.conv2(attn)))
+        attn = F.softmax(self.conv3(attn), dim=1)
+
+        # return ((fusecat * attn).sum(1)).unsqueeze(1)
+        return ((x * attn).sum(1)).unsqueeze(1)
+
+class _DenseLayer(nn.Sequential):
+    def __init__(self, input_features, out_features):
+        super(_DenseLayer, self).__init__()
+
+        # self.add_module('relu2', nn.ReLU(inplace=True)),
+        self.add_module('conv1', nn.Conv2d(input_features, out_features,
+                                           kernel_size=3, stride=1, padding=2, bias=True)),
+        self.add_module('norm1', nn.BatchNorm2d(out_features)),
+        self.add_module('relu1', nn.ReLU(inplace=True)),
+        self.add_module('conv2', nn.Conv2d(out_features, out_features,
+                                           kernel_size=3, stride=1, bias=True)),
+        self.add_module('norm2', nn.BatchNorm2d(out_features))
+
+    def forward(self, x):
+        x1, x2 = x
+
+        new_features = super(_DenseLayer, self).forward(F.relu(x1))  # F.relu()
+        # if new_features.shape[-1]!=x2.shape[-1]:
+        #     new_features =F.interpolate(new_features,size=(x2.shape[2],x2.shape[-1]), mode='bicubic',
+        #                                 align_corners=False)
+        return 0.5 * (new_features + x2), x2
+
+
+class _DenseBlock(nn.Sequential):
+    def __init__(self, num_layers, input_features, out_features):
+        super(_DenseBlock, self).__init__()
+        for i in range(num_layers):
+            layer = _DenseLayer(input_features, out_features)
+            self.add_module('denselayer%d' % (i + 1), layer)
+            input_features = out_features
+
+
+class UpConvBlock(nn.Module):
+    def __init__(self, in_features, up_scale):
+        super(UpConvBlock, self).__init__()
+        self.up_factor = 2
+        self.constant_features = 16
+
+        layers = self.make_deconv_layers(in_features, up_scale)
+        assert layers is not None, layers
+        self.features = nn.Sequential(*layers)
+
+    def make_deconv_layers(self, in_features, up_scale):
+        layers = []
+        all_pads=[0,0,1,3,7]
+        for i in range(up_scale):
+            kernel_size = 2 ** up_scale
+            pad = all_pads[up_scale]  # kernel_size-1
+            out_features = self.compute_out_features(i, up_scale)
+            layers.append(nn.Conv2d(in_features, out_features, 1))
+            layers.append(nn.ReLU(inplace=True))
+            layers.append(nn.ConvTranspose2d(
+                out_features, out_features, kernel_size, stride=2, padding=pad))
+            in_features = out_features
+        return layers
+
+    def compute_out_features(self, idx, up_scale):
+        return 1 if idx == up_scale - 1 else self.constant_features
+
+    def forward(self, x):
+        return self.features(x)
+
+
+class SingleConvBlock(nn.Module):
+    def __init__(self, in_features, out_features, stride,
+                 use_bs=True
+                 ):
+        super(SingleConvBlock, self).__init__()
+        self.use_bn = use_bs
+        self.conv = nn.Conv2d(in_features, out_features, 1, stride=stride,
+                              bias=True)
+        self.bn = nn.BatchNorm2d(out_features)
+
+    def forward(self, x):
+        x = self.conv(x)
+        if self.use_bn:
+            x = self.bn(x)
+        return x
+
+
+class DoubleConvBlock(nn.Module):
+    def __init__(self, in_features, mid_features,
+                 out_features=None,
+                 stride=1,
+                 use_act=True):
+        super(DoubleConvBlock, self).__init__()
+
+        self.use_act = use_act
+        if out_features is None:
+            out_features = mid_features
+        self.conv1 = nn.Conv2d(in_features, mid_features,
+                               3, padding=1, stride=stride)
+        self.bn1 = nn.BatchNorm2d(mid_features)
+        self.conv2 = nn.Conv2d(mid_features, out_features, 3, padding=1)
+        self.bn2 = nn.BatchNorm2d(out_features)
+        self.relu = nn.ReLU(inplace=True)
+
+    def forward(self, x):
+        x = self.conv1(x)
+        x = self.bn1(x)
+        x = self.relu(x)
+        x = self.conv2(x)
+        x = self.bn2(x)
+        if self.use_act:
+            x = self.relu(x)
+        return x
+
+
+class DexiNed(nn.Module):
+    """ Definition of the DXtrem network. """
+
+    def __init__(self):
+        super(DexiNed, self).__init__()
+        self.block_1 = DoubleConvBlock(3, 32, 64, stride=2,)
+        self.block_2 = DoubleConvBlock(64, 128, use_act=False)
+        self.dblock_3 = _DenseBlock(2, 128, 256) # [128,256,100,100]
+        self.dblock_4 = _DenseBlock(3, 256, 512)
+        self.dblock_5 = _DenseBlock(3, 512, 512)
+        self.dblock_6 = _DenseBlock(3, 512, 256)
+        self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
+
+        # left skip connections, figure in Journal
+        self.side_1 = SingleConvBlock(64, 128, 2)
+        self.side_2 = SingleConvBlock(128, 256, 2)
+        self.side_3 = SingleConvBlock(256, 512, 2)
+        self.side_4 = SingleConvBlock(512, 512, 1)
+        self.side_5 = SingleConvBlock(512, 256, 1) # Sory I forget to comment this line :(
+
+        # right skip connections, figure in Journal paper
+        self.pre_dense_2 = SingleConvBlock(128, 256, 2)
+        self.pre_dense_3 = SingleConvBlock(128, 256, 1)
+        self.pre_dense_4 = SingleConvBlock(256, 512, 1)
+        self.pre_dense_5 = SingleConvBlock(512, 512, 1)
+        self.pre_dense_6 = SingleConvBlock(512, 256, 1)
+
+
+        self.up_block_1 = UpConvBlock(64, 1)
+        self.up_block_2 = UpConvBlock(128, 1)
+        self.up_block_3 = UpConvBlock(256, 2)
+        self.up_block_4 = UpConvBlock(512, 3)
+        self.up_block_5 = UpConvBlock(512, 4)
+        self.up_block_6 = UpConvBlock(256, 4)
+        self.block_cat = SingleConvBlock(6, 1, stride=1, use_bs=False) # hed fusion method
+        # self.block_cat = CoFusion(6,6)# cats fusion method
+
+
+        self.apply(weight_init)
+
+    def slice(self, tensor, slice_shape):
+        t_shape = tensor.shape
+        height, width = slice_shape
+        if t_shape[-1]!=slice_shape[-1]:
+            new_tensor = F.interpolate(
+                tensor, size=(height, width), mode='bicubic',align_corners=False)
+        else:
+            new_tensor=tensor
+        # tensor[..., :height, :width]
+        return new_tensor
+
+    def forward(self, x):
+        assert x.ndim == 4, x.shape
+
+        # Block 1
+        block_1 = self.block_1(x)
+        block_1_side = self.side_1(block_1)
+
+        # Block 2
+        block_2 = self.block_2(block_1)
+        block_2_down = self.maxpool(block_2)
+        block_2_add = block_2_down + block_1_side
+        block_2_side = self.side_2(block_2_add)
+
+        # Block 3
+        block_3_pre_dense = self.pre_dense_3(block_2_down)
+        block_3, _ = self.dblock_3([block_2_add, block_3_pre_dense])
+        block_3_down = self.maxpool(block_3) # [128,256,50,50]
+        block_3_add = block_3_down + block_2_side
+        block_3_side = self.side_3(block_3_add)
+
+        # Block 4
+        block_2_resize_half = self.pre_dense_2(block_2_down)
+        block_4_pre_dense = self.pre_dense_4(block_3_down+block_2_resize_half)
+        block_4, _ = self.dblock_4([block_3_add, block_4_pre_dense])
+        block_4_down = self.maxpool(block_4)
+        block_4_add = block_4_down + block_3_side
+        block_4_side = self.side_4(block_4_add)
+
+        # Block 5
+        block_5_pre_dense = self.pre_dense_5(
+            block_4_down) #block_5_pre_dense_512 +block_4_down
+        block_5, _ = self.dblock_5([block_4_add, block_5_pre_dense])
+        block_5_add = block_5 + block_4_side
+
+        # Block 6
+        block_6_pre_dense = self.pre_dense_6(block_5)
+        block_6, _ = self.dblock_6([block_5_add, block_6_pre_dense])
+
+        # upsampling blocks
+        out_1 = self.up_block_1(block_1)
+        out_2 = self.up_block_2(block_2)
+        out_3 = self.up_block_3(block_3)
+        out_4 = self.up_block_4(block_4)
+        out_5 = self.up_block_5(block_5)
+        out_6 = self.up_block_6(block_6)
+        results = [out_1, out_2, out_3, out_4, out_5, out_6]
+
+        # concatenate multiscale outputs
+        block_cat = torch.cat(results, dim=1)  # Bx6xHxW
+        block_cat = self.block_cat(block_cat)  # Bx1xHxW
+
+        # return results
+        results.append(block_cat)
+        return results
+
+
+if __name__ == '__main__':
+    batch_size = 8
+    img_height = 352
+    img_width = 352
+
+    # device = "cuda" if torch.cuda.is_available() else "cpu"
+    device = "cpu"
+    input = torch.rand(batch_size, 3, img_height, img_width).to(device)
+    # target = torch.rand(batch_size, 1, img_height, img_width).to(device)
+    print(f"input shape: {input.shape}")
+    model = DexiNed().to(device)
+    output = model(input)
+    print(f"output shapes: {[t.shape for t in output]}")
+
+    # for i in range(20000):
+    #     print(i)
+    #     output = model(input)
+    #     loss = nn.MSELoss()(output[-1], target)
+    #     loss.backward()

requirements.txt

@@ -1,3 +1,8 @@
-altair
-pandas
-streamlit
+streamlit>=1.0
+torch>=1.4
+opencv-python-headless>=4.5
+matplotlib
+kornia
+numpy
+h5py
+Pillow

res/BIPED/DexiNed-a/eval_bdry.txt

@@ -0,0 +1 @@
+      0.68   0.893704   0.891902   0.892802     0.8983    0.89665   0.897474    0.93966

res/BIPED/DexiNed-a/eval_bdry_img.txt

@@ -0,0 +1,50 @@
+         1       0.67   0.918017   0.921663   0.919836
+         2       0.67   0.924182   0.884619   0.903968
+         3       0.68   0.829315   0.817614   0.823423
+         4        0.7   0.936372    0.90522   0.920533
+         5       0.72   0.893542   0.864391   0.878725
+         6        0.7   0.879453    0.88686   0.883141
+         7       0.67   0.929135   0.856629    0.89141
+         8       0.67   0.913886    0.87818   0.895677
+         9       0.71   0.873921   0.895068   0.884368
+        10        0.7   0.916716   0.915163   0.915939
+        11       0.67   0.920275   0.888373   0.904043
+        12       0.65   0.875954   0.905624   0.890542
+        13   0.656061   0.941063   0.890717   0.915198
+        14   0.703737   0.887589   0.883706   0.885643
+        15       0.62   0.945175   0.912697   0.928652
+        16       0.72   0.904389   0.892603   0.898457
+        17       0.73   0.932328   0.898245   0.914969
+        18   0.647374   0.911589   0.891872   0.901623
+        19       0.69    0.94388   0.946975   0.945425
+        20   0.663939   0.941802   0.921463   0.931521
+        21       0.73   0.973881   0.853938   0.909974
+        22       0.68   0.921542   0.927196    0.92436
+        23       0.62   0.924614   0.937149   0.930839
+        24       0.69   0.936955   0.843424   0.887733
+        25        0.7   0.912016   0.920638   0.916307
+        26       0.68   0.905477   0.888534   0.896925
+        27       0.66   0.865637   0.915651   0.889942
+        28       0.75   0.892646   0.908084   0.900299
+        29       0.68   0.935828   0.915717   0.925663
+        30       0.71   0.913648   0.890326   0.901836
+        31       0.74   0.904806   0.863486   0.883664
+        32       0.59    0.68313   0.921634   0.784658
+        33       0.71   0.838473   0.873585   0.855669
+        34       0.65   0.902898   0.914365   0.908595
+        35       0.68   0.913161   0.910422   0.911789
+        36       0.63    0.90569   0.893079    0.89934
+        37       0.69   0.887816   0.892947   0.890374
+        38       0.59   0.845585   0.914471    0.87868
+        39       0.71   0.908513   0.906695   0.907603
+        40       0.67    0.94974    0.88582   0.916667
+        41       0.71   0.889896   0.949489   0.918727
+        42       0.41   0.849084   0.874457   0.861584
+        43        0.6   0.805666   0.862249   0.832998
+        44        0.7   0.900068   0.908481   0.904255
+        45       0.78   0.844644   0.915197   0.878506
+        46       0.76   0.891072   0.889334   0.890202
+        47       0.59   0.940304   0.940185   0.940244
+        48       0.55   0.916852   0.905893    0.91134
+        49       0.56   0.930273   0.875427   0.902018
+        50        0.5   0.883004   0.914133   0.898299

res/BIPED/DexiNed-a/eval_bdry_thr.txt

@@ -0,0 +1,99 @@
+      0.01   0.988175   0.207782   0.343366
+      0.02   0.986216   0.253632   0.403495
+      0.03    0.98531   0.280848   0.437106
+      0.04   0.984197   0.317963   0.480644
+      0.05    0.98361    0.34075   0.506154
+      0.06    0.98274   0.372731   0.540473
+      0.07   0.982081   0.392988   0.561348
+      0.08   0.980956   0.421624   0.589762
+      0.09   0.980192   0.439719   0.607093
+       0.1   0.979071   0.465276   0.630788
+      0.11   0.978019   0.489119    0.65211
+      0.12   0.977452   0.504122   0.665178
+      0.13   0.976534   0.525536   0.683329
+      0.14   0.975905   0.538951    0.69441
+      0.15   0.975019   0.557785   0.709616
+      0.16   0.974403   0.569401   0.718777
+      0.17   0.973414   0.585647   0.731308
+      0.18   0.972738   0.595669   0.738877
+      0.19   0.971748   0.609387   0.749045
+       0.2   0.971205   0.617662     0.7551
+      0.21   0.970467   0.629194   0.763426
+      0.22   0.969727   0.639676   0.770859
+      0.23   0.969201   0.646257   0.775449
+      0.24   0.968419   0.655424   0.781757
+      0.25     0.9679   0.661159   0.785651
+      0.26   0.967156   0.669391   0.791185
+      0.27   0.966609   0.674612   0.794635
+      0.28   0.965865   0.681981    0.79947
+      0.29   0.965355   0.686676    0.80251
+       0.3   0.964581   0.693399   0.806813
+      0.31   0.963723   0.699708   0.810764
+      0.32   0.963135   0.703819   0.813307
+      0.33   0.962263   0.709873    0.81702
+      0.34   0.961615   0.713707   0.819319
+      0.35   0.960661   0.719355   0.822678
+      0.36   0.959952   0.723046   0.824826
+      0.37   0.958813   0.728583   0.827991
+      0.38   0.958129   0.732161    0.83004
+      0.39   0.957105   0.737354    0.83298
+       0.4   0.956477    0.74065    0.83484
+      0.41   0.955464   0.745614   0.837595
+      0.42    0.95444    0.75057   0.840316
+      0.43   0.953731   0.753903   0.842125
+      0.44    0.95268   0.758965   0.844861
+      0.45   0.951934   0.762367   0.846669
+      0.46   0.950701   0.767516   0.849343
+      0.47    0.94986    0.77107   0.851177
+      0.48   0.948544   0.776395   0.853879
+      0.49   0.947588   0.780009   0.855671
+       0.5   0.946131    0.78556   0.858401
+      0.51   0.944459   0.791298   0.861122
+      0.52     0.9432   0.795149    0.86287
+      0.53   0.941287   0.801239   0.865635
+      0.54   0.939963    0.80534   0.867459
+      0.55   0.937768   0.811506    0.87008
+      0.56   0.936222   0.815705   0.871818
+      0.57   0.933742   0.822284   0.874476
+      0.58   0.932102   0.826791   0.876294
+      0.59   0.929411   0.833635   0.878921
+       0.6   0.927518     0.8385   0.880766
+      0.61   0.924376   0.845633   0.883253
+      0.62   0.920763    0.85294   0.885555
+      0.63   0.918176   0.857962   0.887049
+      0.64   0.913762   0.865697   0.889081
+      0.65   0.910492   0.870926   0.890269
+      0.66   0.904977   0.878862   0.891729
+      0.67   0.900806   0.884203   0.892427
+      0.68   0.893704   0.891902   0.892802
+      0.69    0.88838   0.897135   0.892736
+       0.7   0.879576   0.904908   0.892062
+      0.71   0.869423     0.9122   0.890298
+      0.72   0.861751   0.917011   0.888523
+      0.73   0.849116   0.924042   0.884996
+      0.74   0.839958   0.928623   0.882068
+      0.75   0.824696   0.935411   0.876572
+      0.76    0.81359   0.939705   0.872111
+      0.77   0.795146   0.945445   0.863806
+      0.78   0.782052   0.949373   0.857628
+      0.79   0.759937   0.954739   0.846272
+       0.8   0.743715   0.958184   0.837436
+      0.81   0.717211   0.962967   0.822116
+      0.82   0.687891   0.967594   0.804114
+      0.83    0.66603   0.970528   0.789952
+      0.84   0.630251   0.975011   0.765609
+      0.85   0.603783   0.977975   0.746618
+      0.86   0.559587   0.982141   0.712958
+      0.87   0.526596   0.985282   0.686359
+      0.88   0.470019    0.98926   0.637261
+      0.89   0.427076   0.991085   0.596926
+       0.9   0.355622   0.993421   0.523753
+      0.91   0.278522   0.995622   0.435276
+      0.92   0.225169   0.996738   0.367351
+      0.93   0.145354    0.99803   0.253752
+      0.94  0.0959961    0.99866   0.175155
+      0.95  0.0390728   0.998928  0.0752041
+      0.96  0.0163508    0.99861  0.0321749
+      0.97 0.00220779   0.998104 0.00440584
+      0.98 0.000103649          1 0.000207277
+      0.99          0          0          0

res/BIPED/DexiNed-a/ios_res.txt

@@ -0,0 +1,51 @@
+name,oisT1,oisR1,oisP1,oisF1
+RGB_008,0.6700,0.9180,0.9217,0.9198
+RGB_010,0.6700,0.9242,0.8846,0.9040
+RGB_017,0.6800,0.8293,0.8176,0.8234
+RGB_025,0.7000,0.9364,0.9052,0.9205
+RGB_029,0.7200,0.8935,0.8644,0.8787
+RGB_033,0.7000,0.8795,0.8869,0.8831
+RGB_035,0.6700,0.9291,0.8566,0.8914
+RGB_039,0.6700,0.9139,0.8782,0.8957
+RGB_042,0.7100,0.8739,0.8951,0.8844
+RGB_052,0.7000,0.9167,0.9152,0.9159
+RGB_056,0.6700,0.9203,0.8884,0.9040
+RGB_057,0.6500,0.8760,0.9056,0.8905
+RGB_059,0.6561,0.9411,0.8907,0.9152
+RGB_066,0.7037,0.8876,0.8837,0.8856
+RGB_067,0.6200,0.9452,0.9127,0.9287
+RGB_070,0.7200,0.9044,0.8926,0.8985
+RGB_071,0.7300,0.9323,0.8982,0.9150
+RGB_075,0.6474,0.9116,0.8919,0.9016
+RGB_081,0.6900,0.9439,0.9470,0.9454
+RGB_083,0.6639,0.9418,0.9215,0.9315
+RGB_090,0.7300,0.9739,0.8539,0.9100
+RGB_092,0.6800,0.9215,0.9272,0.9244
+RGB_094,0.6200,0.9246,0.9371,0.9308
+RGB_098,0.6900,0.9370,0.8434,0.8877
+RGB_102,0.7000,0.9120,0.9206,0.9163
+RGB_107,0.6800,0.9055,0.8885,0.8969
+RGB_111,0.6600,0.8656,0.9157,0.8899
+RGB_120,0.7500,0.8926,0.9081,0.9003
+RGB_124,0.6800,0.9358,0.9157,0.9257
+RGB_132,0.7100,0.9136,0.8903,0.9018
+RGB_138,0.7400,0.9048,0.8635,0.8837
+RGB_139,0.5900,0.6831,0.9216,0.7847
+RGB_141,0.7100,0.8385,0.8736,0.8557
+RGB_145,0.6500,0.9029,0.9144,0.9086
+RGB_160,0.6800,0.9132,0.9104,0.9118
+RGB_170,0.6300,0.9057,0.8931,0.8993
+RGB_181,0.6900,0.8878,0.8929,0.8904
+RGB_182,0.5900,0.8456,0.9145,0.8787
+RGB_190,0.7100,0.9085,0.9067,0.9076
+RGB_192,0.6700,0.9497,0.8858,0.9167
+RGB_206,0.7100,0.8899,0.9495,0.9187
+RGB_212,0.4100,0.8491,0.8745,0.8616
+RGB_214,0.6000,0.8057,0.8622,0.8330
+RGB_226,0.7000,0.9001,0.9085,0.9043
+RGB_228,0.7800,0.8446,0.9152,0.8785
+RGB_234,0.7600,0.8911,0.8893,0.8902
+RGB_237,0.5900,0.9403,0.9402,0.9402
+RGB_247,0.5500,0.9169,0.9059,0.9113
+RGB_248,0.5600,0.9303,0.8754,0.9020
+RGB_254,0.5000,0.8830,0.9141,0.8983

res/BIPED/DexiNed-f/eval_bdry.txt

@@ -0,0 +1 @@
+      0.72   0.893459   0.896543   0.894998   0.902943    0.89788   0.900405   0.926588

res/BIPED/DexiNed-f/eval_bdry_img.txt

@@ -0,0 +1,50 @@
+         1       0.69   0.926056   0.915888   0.920944
+         2       0.71   0.919448   0.908713   0.914049
+         3       0.75   0.843854   0.831785   0.837776
+         4        0.8   0.936259   0.909764   0.922821
+         5       0.77    0.88401   0.856038   0.869799
+         6       0.79    0.87562   0.902286   0.888753
+         7       0.65   0.949605   0.829958   0.885759
+         8       0.71   0.894801   0.891634   0.893214
+         9   0.744141    0.89899   0.914359   0.906609
+        10       0.68   0.934196   0.898731   0.916121
+        11       0.66   0.927261   0.887812   0.907107
+        12       0.69   0.860233   0.925554   0.891699
+        13       0.66   0.961541   0.904682   0.932245
+        14       0.71    0.90251    0.87612   0.889119
+        15       0.68   0.927315   0.940563   0.933892
+        16       0.77   0.910788   0.893992   0.902311
+        17       0.75   0.934857   0.896983   0.915529
+        18       0.65   0.918032   0.903552   0.910735
+        19       0.74   0.945621   0.952114   0.948856
+        20       0.61   0.958399   0.923796   0.940779
+        21       0.81   0.968809   0.865592   0.914296
+        22       0.74   0.932564   0.916397    0.92441
+        23       0.64   0.912742    0.94139   0.926845
+        24       0.71   0.934661   0.861214   0.896436
+        25       0.71   0.916226   0.924688   0.920438
+        26       0.72   0.915938   0.881975   0.898636
+        27       0.62   0.888638   0.897102    0.89285
+        28       0.78   0.926264   0.905505   0.915767
+        29       0.69   0.929126   0.914336   0.921672
+        30       0.73   0.921188     0.8909   0.905791
+        31       0.79   0.917942   0.851855   0.883664
+        32       0.55   0.686601   0.934876   0.791731
+        33       0.71   0.863806   0.866542   0.865172
+        34       0.61   0.913651   0.904125   0.908863
+        35       0.71   0.905047    0.90312   0.904082
+        36       0.59    0.92391    0.89769   0.910611
+        37       0.67   0.906366   0.879008   0.892477
+        38       0.55   0.848673   0.905456   0.876145
+        39       0.76   0.886487   0.902989   0.894662
+        40       0.68   0.945881   0.895856   0.920189
+        41       0.72   0.903101   0.937613   0.920033
+        42       0.16   0.844707   0.880088   0.862035
+        43       0.55   0.848311   0.854467   0.851378
+        44       0.73   0.890069   0.907548   0.898724
+        45       0.84   0.836801    0.91615   0.874679
+        46       0.78   0.909175   0.874802   0.891657
+        47        0.5   0.921918   0.944137   0.932895
+        48       0.32   0.921204    0.90472   0.912888
+        49       0.44    0.94737   0.889883   0.917727
+        50       0.25    0.89233   0.915361   0.903699

res/BIPED/DexiNed-f/eval_bdry_thr.txt

@@ -0,0 +1,99 @@
+      0.01   0.962888   0.692543    0.80564
+      0.02   0.960933    0.71092   0.817232
+      0.03    0.96017   0.717425   0.821235
+      0.04   0.959307   0.724555   0.825567
+      0.05   0.958872   0.728389   0.827889
+      0.06   0.958251   0.733298   0.830817
+      0.07   0.957905   0.736193   0.832541
+      0.08    0.95731   0.740079   0.834794
+      0.09   0.956905   0.742451   0.836147
+       0.1   0.956386   0.745858   0.838103
+      0.11   0.955846     0.7491   0.839938
+      0.12   0.955531    0.75113    0.84109
+      0.13   0.955084   0.754159   0.842812
+      0.14   0.954775   0.756175   0.843949
+      0.15   0.954276   0.759107   0.845576
+      0.16    0.95394   0.761087    0.84667
+      0.17    0.95336   0.764057   0.848275
+      0.18   0.952929   0.766236   0.849446
+      0.19   0.951941   0.769254   0.850902
+       0.2   0.951526   0.770822   0.851695
+      0.21    0.95103   0.773054   0.852856
+      0.22   0.950612   0.775275   0.854037
+      0.23   0.950318   0.776742   0.854808
+      0.24   0.949884   0.778958   0.855971
+      0.25   0.949554    0.78036   0.856683
+      0.26   0.949115   0.782458   0.857767
+      0.27   0.948809   0.783828   0.858464
+      0.28   0.948326   0.785945   0.859534
+      0.29   0.948024   0.787363   0.860256
+       0.3   0.947534   0.789401   0.861269
+      0.31   0.947056   0.791569    0.86236
+      0.32   0.946731   0.793041   0.863098
+      0.33   0.946165   0.795214   0.864147
+      0.34   0.945812   0.796697   0.864874
+      0.35   0.945213   0.798981   0.865967
+      0.36   0.944761   0.800546   0.866696
+      0.37   0.943906   0.802928   0.867728
+      0.38   0.943315   0.804466   0.868376
+      0.39   0.942604   0.806703   0.869374
+       0.4   0.942171   0.808189   0.870052
+      0.41   0.941488   0.810427   0.871055
+      0.42   0.940734   0.812654   0.872016
+      0.43   0.940212   0.814186   0.872673
+      0.44   0.939358   0.816525   0.873645
+      0.45   0.938869   0.818122   0.874347
+      0.46   0.938006   0.820602   0.875385
+      0.47   0.937369   0.822269   0.876055
+      0.48   0.936494    0.82485   0.877134
+      0.49   0.935873   0.826621   0.877861
+       0.5   0.934833   0.829298   0.878909
+      0.51   0.933855   0.832137   0.880067
+      0.52   0.933124   0.833972   0.880766
+      0.53   0.931935   0.836982    0.88191
+      0.54   0.931025    0.83903   0.882637
+      0.55   0.929619   0.842154   0.883728
+      0.56   0.928601   0.844236   0.884411
+      0.57   0.926943   0.847476    0.88543
+      0.58    0.92591   0.849679   0.886158
+      0.59   0.924312   0.852897    0.88717
+       0.6   0.923159   0.855004   0.887776
+      0.61   0.921425   0.858361   0.888776
+      0.62   0.919475   0.861817   0.889713
+      0.63   0.918119   0.864269   0.890381
+      0.64   0.915871   0.867947   0.891265
+      0.65   0.914291   0.870452   0.891833
+      0.66   0.911733   0.874354   0.892652
+      0.67   0.909824   0.877017    0.89312
+      0.68   0.906768   0.881267   0.893836
+      0.69   0.904447   0.884212   0.894215
+       0.7   0.900691   0.888645   0.894628
+      0.71   0.896555   0.893313   0.894931
+      0.72   0.893459   0.896543   0.894998
+      0.73   0.888287   0.901458   0.894824
+      0.74   0.884496   0.904712    0.89449
+      0.75   0.877767   0.909729   0.893462
+      0.76   0.872756   0.913224   0.892531
+      0.77   0.864504   0.918394   0.890635
+      0.78   0.858398   0.922011   0.889068
+      0.79   0.848066   0.927246    0.88589
+       0.8   0.840315   0.930758   0.883227
+      0.81   0.826772   0.935883    0.87795
+      0.82    0.81089   0.941024   0.871123
+      0.83   0.798715   0.944474   0.865501
+      0.84   0.777873   0.949745    0.85526
+      0.85   0.761862    0.95316   0.846842
+      0.86   0.734225   0.957972   0.831306
+      0.87   0.712997   0.961176   0.818691
+      0.88   0.675414   0.965892   0.794949
+      0.89   0.646561   0.969267    0.77569
+       0.9   0.595381   0.974561    0.73918
+      0.91   0.532278   0.979687   0.689785
+      0.92   0.482637   0.983132   0.647436
+      0.93   0.396158   0.987503   0.565467
+      0.94   0.331673   0.989725   0.496846
+      0.95   0.229612   0.992629   0.372953
+      0.96   0.161409   0.994562   0.277742
+      0.97  0.0714786   0.996692   0.133391
+      0.98  0.0295587   0.996888  0.0574149
+      0.99 0.00279374   0.997647 0.00557188

res/BIPED/DexiNed-f/ios_res.txt

@@ -0,0 +1,51 @@
+name,oisT1,oisR1,oisP1,oisF1
+RGB_008,0.6900,0.9261,0.9159,0.9209
+RGB_010,0.7100,0.9194,0.9087,0.9140
+RGB_017,0.7500,0.8439,0.8318,0.8378
+RGB_025,0.8000,0.9363,0.9098,0.9228
+RGB_029,0.7700,0.8840,0.8560,0.8698
+RGB_033,0.7900,0.8756,0.9023,0.8888
+RGB_035,0.6500,0.9496,0.8300,0.8858
+RGB_039,0.7100,0.8948,0.8916,0.8932
+RGB_042,0.7441,0.8990,0.9144,0.9066
+RGB_052,0.6800,0.9342,0.8987,0.9161
+RGB_056,0.6600,0.9273,0.8878,0.9071
+RGB_057,0.6900,0.8602,0.9256,0.8917
+RGB_059,0.6600,0.9615,0.9047,0.9322
+RGB_066,0.7100,0.9025,0.8761,0.8891
+RGB_067,0.6800,0.9273,0.9406,0.9339
+RGB_070,0.7700,0.9108,0.8940,0.9023
+RGB_071,0.7500,0.9349,0.8970,0.9155
+RGB_075,0.6500,0.9180,0.9036,0.9107
+RGB_081,0.7400,0.9456,0.9521,0.9489
+RGB_083,0.6100,0.9584,0.9238,0.9408
+RGB_090,0.8100,0.9688,0.8656,0.9143
+RGB_092,0.7400,0.9326,0.9164,0.9244
+RGB_094,0.6400,0.9127,0.9414,0.9268
+RGB_098,0.7100,0.9347,0.8612,0.8964
+RGB_102,0.7100,0.9162,0.9247,0.9204
+RGB_107,0.7200,0.9159,0.8820,0.8986
+RGB_111,0.6200,0.8886,0.8971,0.8928
+RGB_120,0.7800,0.9263,0.9055,0.9158
+RGB_124,0.6900,0.9291,0.9143,0.9217
+RGB_132,0.7300,0.9212,0.8909,0.9058
+RGB_138,0.7900,0.9179,0.8519,0.8837
+RGB_139,0.5500,0.6866,0.9349,0.7917
+RGB_141,0.7100,0.8638,0.8665,0.8652
+RGB_145,0.6100,0.9137,0.9041,0.9089
+RGB_160,0.7100,0.9050,0.9031,0.9041
+RGB_170,0.5900,0.9239,0.8977,0.9106
+RGB_181,0.6700,0.9064,0.8790,0.8925
+RGB_182,0.5500,0.8487,0.9055,0.8761
+RGB_190,0.7600,0.8865,0.9030,0.8947
+RGB_192,0.6800,0.9459,0.8959,0.9202
+RGB_206,0.7200,0.9031,0.9376,0.9200
+RGB_212,0.1600,0.8447,0.8801,0.8620
+RGB_214,0.5500,0.8483,0.8545,0.8514
+RGB_226,0.7300,0.8901,0.9075,0.8987
+RGB_228,0.8400,0.8368,0.9162,0.8747
+RGB_234,0.7800,0.9092,0.8748,0.8917
+RGB_237,0.5000,0.9219,0.9441,0.9329
+RGB_247,0.3200,0.9212,0.9047,0.9129
+RGB_248,0.4400,0.9474,0.8899,0.9177
+RGB_254,0.2500,0.8923,0.9154,0.9037

res/MDBD/DexiNed-a/eval_bdry.txt

@@ -0,0 +1 @@
+  0.597374   0.885572   0.901677   0.893552   0.892316   0.912223    0.90216   0.950624

res/MDBD/DexiNed-a/eval_bdry_img.txt

@@ -0,0 +1,20 @@
+         1       0.56   0.901122   0.887133   0.894073
+         2        0.6    0.81555   0.891492   0.851832
+         3       0.67   0.896541   0.889383   0.892947
+         4       0.56   0.868946   0.917992   0.892796
+         5       0.69   0.913616   0.856855   0.884325
+         6       0.49   0.934683   0.921958   0.928277
+         7       0.74   0.883474   0.900804   0.892055
+         8       0.65   0.868302   0.823076   0.845084
+         9       0.67   0.924908   0.897714   0.911108
+        10       0.64    0.86446   0.916706   0.889817
+        11       0.53    0.86169   0.886498   0.873918
+        12       0.56   0.878293   0.915788   0.896649
+        13       0.46   0.856196   0.964041   0.906924
+        14       0.56   0.921281   0.928259   0.924757
+        15       0.54    0.86711   0.929321   0.897138
+        16       0.59   0.918537    0.93561   0.926995
+        17       0.63   0.885477   0.904377   0.894827
+        18   0.563434   0.908339   0.953304   0.930279
+        19       0.54   0.914219    0.94903   0.931299
+        20       0.53     0.9212   0.958346   0.939406

res/MDBD/DexiNed-a/eval_bdry_thr.txt

@@ -0,0 +1,99 @@
+      0.01   0.997164    0.45554   0.625383
+      0.02   0.995287    0.48181     0.6493
+      0.03   0.994349    0.49603   0.661881
+      0.04   0.993129   0.518086   0.680944
+      0.05   0.992364   0.531244   0.692025
+      0.06   0.991129   0.549036   0.706633
+      0.07   0.990422   0.559727   0.715242
+      0.08   0.989247   0.573851   0.726353
+      0.09    0.98832    0.58224   0.732783
+       0.1   0.986828   0.593985   0.741595
+      0.11   0.985176    0.60546   0.749995
+      0.12   0.984029   0.612892   0.755333
+      0.13   0.982403   0.623862   0.763117
+      0.14   0.981364   0.631107   0.768195
+      0.15   0.979793   0.641587   0.775416
+      0.16   0.978788   0.648459   0.780095
+      0.17   0.977308   0.658501   0.786838
+      0.18   0.976252   0.664958   0.791083
+      0.19   0.974762   0.674437   0.797254
+       0.2   0.973823   0.680309   0.801025
+      0.21   0.972416   0.688868   0.806444
+      0.22   0.971057   0.697024   0.811531
+      0.23   0.970157   0.702369   0.814825
+      0.24   0.968823   0.710171   0.819575
+      0.25   0.967954   0.715159   0.822572
+      0.26   0.966538   0.722379    0.82681
+      0.27   0.965605   0.727031   0.829505
+      0.28   0.964087   0.733703   0.833264
+      0.29   0.963088   0.738071   0.835698
+       0.3   0.961581   0.744561   0.839269
+      0.31   0.960053   0.750888   0.842685
+      0.32   0.958957   0.755057   0.844879
+      0.33   0.957326   0.761197   0.848069
+      0.34   0.956148   0.765238   0.850107
+      0.35    0.95449   0.771431   0.853252
+      0.36   0.953254    0.77559   0.855293
+      0.37   0.951372   0.781732   0.858249
+      0.38   0.950158   0.785862   0.860236
+      0.39   0.948265   0.792071   0.863159
+       0.4   0.946922   0.796288   0.865097
+      0.41   0.944817   0.802722   0.867992
+      0.42   0.942567   0.809143   0.870773
+      0.43   0.941031   0.813415   0.872582
+      0.44   0.938613   0.820013   0.875314
+      0.45    0.93692    0.82439    0.87706
+      0.46   0.934208   0.830853   0.879505
+      0.47   0.932229   0.835128   0.881011
+      0.48   0.929103   0.841585   0.883181
+      0.49   0.926979   0.845712   0.884483
+       0.5   0.923675   0.852045   0.886415
+      0.51   0.919927   0.858232   0.888009
+      0.52    0.91733   0.862296   0.888962
+      0.53   0.913235   0.868625   0.890372
+      0.54   0.910276   0.872732   0.891109
+      0.55   0.905607    0.87876   0.891982
+      0.56   0.902487   0.882871   0.892571
+      0.57   0.897167   0.888791   0.892959
+      0.58   0.893677   0.892722   0.893199
+      0.59   0.888421   0.898724   0.893543
+       0.6   0.884558   0.902729   0.893551
+      0.61   0.878321   0.908664   0.893235
+      0.62   0.871641   0.914455   0.892535
+      0.63   0.866907   0.918307   0.891867
+      0.64   0.859239   0.923631   0.890272
+      0.65    0.85383   0.927064   0.888941
+      0.66   0.845158   0.932109   0.886507
+      0.67   0.838919   0.935359   0.884518
+      0.68   0.828825   0.940001   0.880919
+      0.69   0.821503   0.942855   0.878006
+       0.7   0.810015   0.947101    0.87321
+      0.71   0.797573   0.951176   0.867628
+      0.72   0.788436   0.953801   0.863271
+      0.73   0.774273   0.957826   0.856324
+      0.74   0.764061   0.960408   0.851056
+      0.75   0.747309   0.964059   0.841958
+      0.76   0.735229   0.966431   0.835124
+      0.77   0.715608   0.969658   0.823484
+      0.78   0.701222   0.971753   0.814614
+      0.79   0.677738   0.975121   0.799677
+       0.8    0.66031   0.977105   0.788062
+      0.81    0.63136   0.980026   0.767971
+      0.82   0.598865    0.98275   0.744219
+      0.83   0.575167    0.98455   0.726132
+      0.84   0.535904   0.987078   0.694663
+      0.85   0.507979   0.988498   0.671091
+      0.86   0.461462   0.990022   0.629504
+      0.87   0.428033   0.991012   0.597846
+      0.88   0.375164     0.9928   0.544551
+      0.89   0.338149   0.993739   0.504595
+       0.9    0.28077   0.994884   0.437946
+      0.91   0.221726   0.996313   0.362728
+      0.92   0.183093   0.996871   0.309366
+      0.93   0.127928   0.998276   0.226792
+      0.94   0.093393    0.99886   0.170815
+      0.95  0.0508334   0.999626   0.096747
+      0.96  0.0296794          1  0.0576479
+      0.97 0.00867657          1  0.0172039
+      0.98  0.0021924          1  0.0043752
+      0.99 1.45917e-05          1 2.91829e-05

res/MDBD/DexiNed-a/ios_res.txt

@@ -0,0 +1,21 @@
+name,oisT1,oisR1,oisP1,oisF1
+4100_left_2268,0.5600,0.9011,0.8871,0.8941
+4101_left_0382,0.6000,0.8156,0.8915,0.8518
+4108_left_0082,0.6700,0.8965,0.8894,0.8929
+4113_left_0082,0.5600,0.8689,0.9180,0.8928
+4114_left_0611,0.6900,0.9136,0.8569,0.8843
+4176_left_0273,0.4900,0.9347,0.9220,0.9283
+4178_left_0052,0.7400,0.8835,0.9008,0.8921
+4179_left_0315,0.6500,0.8683,0.8231,0.8451
+4187_left_0071,0.6700,0.9249,0.8977,0.9111
+4209_left_0034,0.6400,0.8645,0.9167,0.8898
+4377_left_0016,0.5300,0.8617,0.8865,0.8739
+4386_left_0020,0.5600,0.8783,0.9158,0.8966
+4406_left_0030,0.4600,0.8562,0.9640,0.9069
+4523_left_0070,0.5600,0.9213,0.9283,0.9248
+4558_left_0260,0.5400,0.8671,0.9293,0.8971
+4580_left_0165,0.5900,0.9185,0.9356,0.9270
+4603_left_0050,0.6300,0.8855,0.9044,0.8948
+4663_left_0010,0.5634,0.9083,0.9533,0.9303
+4684_left_0010,0.5400,0.9142,0.9490,0.9313
+4685_left_0045,0.5300,0.9212,0.9583,0.9394

res/MDBD/DexiNed-f/eval_bdry.txt

@@ -0,0 +1 @@
+      0.53   0.886613   0.895778   0.891172   0.886553   0.905303    0.89583   0.930116

res/MDBD/DexiNed-f/eval_bdry_img.txt

@@ -0,0 +1,20 @@
+         1       0.51   0.878821   0.890935   0.884836
+         2       0.51   0.838008    0.84884   0.843389
+         3       0.63   0.892969   0.897147   0.895053
+         4   0.499798   0.875027   0.920392   0.897136
+         5       0.68   0.904454   0.863271   0.883383
+         6       0.45   0.932002   0.921072   0.926505
+         7       0.67   0.912025   0.891484   0.901637
+         8       0.62   0.874182   0.831176   0.852137
+         9       0.62   0.919341   0.900196   0.909668
+        10       0.62   0.864278   0.918305   0.890473
+        11       0.49    0.84351   0.871435   0.857245
+        12        0.5   0.877031   0.891973   0.884439
+        13       0.37   0.837118   0.942025   0.886478
+        14       0.46    0.91483   0.912852    0.91384
+        15        0.5   0.845711   0.924797   0.883487
+        16       0.51   0.910663   0.912789   0.911725
+        17        0.6   0.871541   0.896135   0.883667
+        18        0.5   0.902447    0.94623    0.92382
+        19       0.44   0.902223    0.95517   0.927942
+        20       0.42    0.91606   0.957079    0.93612

res/MDBD/DexiNed-f/eval_bdry_thr.txt

@@ -0,0 +1,99 @@
+      0.01   0.965678   0.692439   0.806545
+      0.02   0.958628   0.722879   0.824228
+      0.03   0.956763   0.730901   0.828719
+      0.04   0.955035   0.738735   0.833074
+      0.05   0.954159   0.742615     0.8352
+      0.06   0.953006   0.747683   0.837951
+      0.07   0.952371    0.75073   0.839614
+      0.08   0.951402   0.754995   0.841895
+      0.09    0.95076     0.7577   0.843322
+       0.1   0.949851   0.761252   0.845158
+      0.11   0.948941   0.764879   0.847026
+      0.12   0.948363   0.767223    0.84823
+      0.13   0.947501   0.770643   0.849969
+      0.14   0.946908    0.77311   0.851228
+      0.15   0.945982   0.776707   0.853028
+      0.16   0.945326   0.779021   0.854153
+      0.17   0.944324   0.782726   0.855965
+      0.18   0.943622   0.785122   0.857106
+      0.19   0.942603   0.788671   0.858794
+       0.2   0.941899   0.790932   0.859839
+      0.21    0.94086   0.794267   0.861371
+      0.22   0.939882   0.797898    0.86309
+      0.23   0.939216   0.800267   0.864192
+      0.24   0.938202   0.803735   0.865778
+      0.25   0.937494   0.806175   0.866889
+      0.26   0.936339   0.809779   0.868472
+      0.27   0.935561   0.812184   0.869518
+      0.28   0.934376   0.815838   0.871093
+      0.29   0.933536   0.818237   0.872092
+       0.3   0.932218   0.821922   0.873602
+      0.31   0.930724    0.82549   0.874954
+      0.32   0.929701    0.82789   0.875847
+      0.33   0.928087   0.831446   0.877113
+      0.34   0.926953   0.833929   0.877984
+      0.35   0.925207   0.837501   0.879172
+      0.36    0.92399    0.83988   0.879929
+      0.37   0.922091   0.843496   0.881044
+      0.38   0.920841   0.845898    0.88178
+      0.39   0.918976   0.849412   0.882826
+       0.4   0.917685   0.851842   0.883539
+      0.41   0.915677   0.855602   0.884621
+      0.42   0.913514   0.859323    0.88559
+      0.43   0.911999   0.861899   0.886242
+      0.44   0.909645   0.865832   0.887198
+      0.45   0.907982   0.868393   0.887746
+      0.46   0.905435   0.872228   0.888521
+      0.47   0.903599   0.874846    0.88899
+      0.48   0.900794   0.878956   0.889741
+      0.49   0.898862   0.881509   0.890101
+       0.5   0.895733   0.885508   0.890591
+      0.51   0.892475   0.889491    0.89098
+      0.52   0.890182   0.891997   0.891088
+      0.53   0.886613   0.895778   0.891172
+      0.54   0.884083   0.898142   0.891057
+      0.55   0.880063    0.90178   0.890789
+      0.56   0.877293    0.90423   0.890558
+      0.57   0.872936   0.907771   0.890013
+      0.58   0.869865   0.910329   0.889637
+      0.59   0.865244   0.913996   0.888952
+       0.6   0.862111   0.916521   0.888484
+      0.61   0.856915   0.920525   0.887582
+      0.62   0.851391   0.924404   0.886396
+      0.63   0.847464   0.926962   0.885432
+      0.64   0.841336   0.930631   0.883734
+      0.65      0.837   0.933276    0.88252
+      0.66   0.829985   0.937115   0.880302
+      0.67   0.824815   0.939536   0.878446
+      0.68   0.816661   0.943224   0.875391
+      0.69   0.810776   0.945443   0.872946
+       0.7   0.801784   0.948789   0.869114
+      0.71    0.79181   0.951996   0.864546
+      0.72   0.784888    0.95394   0.861196
+      0.73   0.773639   0.957107   0.855648
+      0.74   0.765556    0.95905   0.851448
+      0.75    0.75246   0.962235   0.844516
+      0.76   0.743163   0.964212   0.839378
+      0.77   0.727774    0.96712   0.830547
+      0.78   0.716703   0.968852   0.823918
+      0.79   0.698734    0.97168   0.812907
+       0.8   0.685435    0.97361   0.804494
+      0.81   0.663805   0.976447    0.79033
+      0.82   0.639137   0.979333   0.773481
+      0.83    0.62099   0.981101   0.760573
+      0.84   0.591026   0.983656   0.738392
+      0.85   0.568244   0.985456   0.720834
+      0.86   0.529949   0.987811   0.689818
+      0.87   0.501772   0.989322   0.665839
+      0.88   0.455247   0.991302    0.62395
+      0.89   0.422051    0.99255   0.592261
+       0.9    0.36763   0.994005   0.536746
+      0.91   0.310516   0.994874   0.473306
+      0.92   0.270527   0.995394   0.425431
+      0.93   0.210046   0.996397   0.346953
+      0.94   0.170149   0.996698   0.290676
+      0.95   0.112359   0.997894   0.201976
+      0.96  0.0767081   0.998212   0.142468
+      0.97  0.0334125   0.999904  0.0646642
+      0.98  0.0136791          1   0.026989
+      0.99 0.00108374          1 0.00216512